diff --git a/doc/LectureNotes/exercisesweek34.ipynb b/doc/LectureNotes/exercisesweek34.ipynb
deleted file mode 100644
index 6a84a1d30..000000000
--- a/doc/LectureNotes/exercisesweek34.ipynb
+++ /dev/null
@@ -1,303 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "a50ea987",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
-    "doconce format html exercisesweek34.do.txt  -->\n",
-    "<!-- dom:TITLE: Exercises week 34 -->"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "291e6015",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "# Exercises week 34\n",
-    "**FYS-STK3155/4155**\n",
-    "\n",
-    "Date: **August 19-23, 2024**"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "141ff111",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "## Exercises\n",
-    "\n",
-    "Here are three possible exercises for week 34"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "32a1fc9e",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "## Exercise 1: Setting up various Python environments\n",
-    "\n",
-    "The first exercise here is of a mere technical art. We want you to have \n",
-    "* git as a version control software and to establish a user account on a provider like GitHub. Other providers like GitLab etc are equally fine. You can also use the University of Oslo [GitHub facilities](https://www.uio.no/tjenester/it/maskin/filer/versjonskontroll/github.html). \n",
-    "\n",
-    "* Install various Python packages\n",
-    "\n",
-    "We will make extensive use of Python as programming language and its\n",
-    "myriad of available libraries.  You will find\n",
-    "IPython/Jupyter notebooks invaluable in your work.  You can run **R**\n",
-    "codes in the Jupyter/IPython notebooks, with the immediate benefit of\n",
-    "visualizing your data. You can also use compiled languages like C++,\n",
-    "Rust, Fortran etc if you prefer. The focus in these lectures will be\n",
-    "on Python.\n",
-    "\n",
-    "If you have Python installed (we recommend Python3) and you feel\n",
-    "pretty familiar with installing different packages, we recommend that\n",
-    "you install the following Python packages via **pip** as \n",
-    "\n",
-    "1. pip install numpy scipy matplotlib ipython scikit-learn sympy pandas pillow \n",
-    "\n",
-    "For **Tensorflow**, we recommend following the instructions in the text of \n",
-    "[Aurelien Geron, Hands‑On Machine Learning with Scikit‑Learn and TensorFlow, O'Reilly](http://shop.oreilly.com/product/0636920052289.do)\n",
-    "\n",
-    "We will come back to **tensorflow** later. \n",
-    "\n",
-    "For Python3, replace **pip** with **pip3**.\n",
-    "\n",
-    "For OSX users we recommend, after having installed Xcode, to\n",
-    "install **brew**. Brew allows for a seamless installation of additional\n",
-    "software via for example \n",
-    "\n",
-    "1. brew install python3\n",
-    "\n",
-    "For Linux users, with its variety of distributions like for example the widely popular Ubuntu distribution,\n",
-    "you can use **pip** as well and simply install Python as \n",
-    "\n",
-    "1. sudo apt-get install python3  (or python for Python2.7)\n",
-    "\n",
-    "If you don't want to perform these operations separately and venture\n",
-    "into the hassle of exploring how to set up dependencies and paths, we\n",
-    "recommend two widely used distrubutions which set up all relevant\n",
-    "dependencies for Python, namely \n",
-    "\n",
-    "* [Anaconda](https://docs.anaconda.com/), \n",
-    "\n",
-    "which is an open source\n",
-    "distribution of the Python and R programming languages for large-scale\n",
-    "data processing, predictive analytics, and scientific computing, that\n",
-    "aims to simplify package management and deployment. Package versions\n",
-    "are managed by the package management system **conda**. \n",
-    "\n",
-    "* [Enthought canopy](https://www.enthought.com/product/canopy/) \n",
-    "\n",
-    "is a Python\n",
-    "distribution for scientific and analytic computing distribution and\n",
-    "analysis environment, available for free and under a commercial\n",
-    "license.\n",
-    "\n",
-    "We recommend using **Anaconda** if you are not too familiar with setting paths in a terminal environment."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "f103f4c1",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "## Exercise 2: making your own data and exploring scikit-learn\n",
-    "\n",
-    "We will generate our own dataset for a function $y(x)$ where $x \\in [0,1]$ and defined by random numbers computed with the uniform distribution. The function $y$ is a quadratic polynomial in $x$ with added stochastic noise according to the normal distribution $\\cal {N}(0,1)$.\n",
-    "The following simple Python instructions define our $x$ and $y$ values (with 100 data points)."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "id": "81c11edc",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
-   "outputs": [],
-   "source": [
-    "import numpy as np\n",
-    "x = np.random.rand(100,1)\n",
-    "y = 2.0+5*x*x+0.1*np.random.randn(100,1)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "970bd604",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "1. Write your own code (following the examples under the [regression notes](https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/chapter1.html)) for computing the parametrization of the data set fitting a second-order polynomial. \n",
-    "\n",
-    "2. Use thereafter **scikit-learn** (see again the examples in the regression slides) and compare with your own code.   \n",
-    "\n",
-    "3. Using scikit-learn, compute also the mean square error, a risk metric corresponding to the expected value of the squared (quadratic) error defined as"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "bda3ed04",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "MSE(\\boldsymbol{y},\\boldsymbol{\\tilde{y}}) = \\frac{1}{n}\n",
-    "\\sum_{i=0}^{n-1}(y_i-\\tilde{y}_i)^2,\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "ca995691",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "and the $R^2$ score function.\n",
-    "If $\\tilde{\\boldsymbol{y}}_i$ is the predicted value of the $i-th$ sample and $y_i$ is the corresponding true value, then the score $R^2$ is defined as"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "27cbfeed",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "R^2(\\boldsymbol{y}, \\tilde{\\boldsymbol{y}}) = 1 - \\frac{\\sum_{i=0}^{n - 1} (y_i - \\tilde{y}_i)^2}{\\sum_{i=0}^{n - 1} (y_i - \\bar{y})^2},\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "9e063f6d",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "where we have defined the mean value  of $\\boldsymbol{y}$ as"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "e582c2bb",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\bar{y} =  \\frac{1}{n} \\sum_{i=0}^{n - 1} y_i.\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "59e0ed7a",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "You can use the functionality included in scikit-learn. If you feel for it, you can use your own program and define functions which compute the above two functions. \n",
-    "Discuss the meaning of these results. Try also to vary the coefficient in front of the added stochastic noise term and discuss the quality of the fits."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "051e71aa",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "## Exercise 3: Split data in test and training data\n",
-    "\n",
-    "In this exercise we want you to to compute the MSE for the training\n",
-    "data and the test data as function of the complexity of a polynomial,\n",
-    "that is the degree of a given polynomial.\n",
-    "\n",
-    "The aim is to reproduce Figure 2.11 of [Hastie et al](https://github.com/CompPhysics/MLErasmus/blob/master/doc/Textbooks/elementsstat.pdf).\n",
-    "\n",
-    "Our data is defined by $x\\in [-3,3]$ with a total of for example $n=100$ data points. You should try to vary the number of data points $n$ in your analysis."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "id": "c82d8b64",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
-   "outputs": [],
-   "source": [
-    "np.random.seed()\n",
-    "n = 100\n",
-    "# Make data set.\n",
-    "x = np.linspace(-3, 3, n).reshape(-1, 1)\n",
-    "y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "3e810bba",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "where $y$ is the function we want to fit with a given polynomial."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "c1198fce",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "**a)**\n",
-    "Write a first code which sets up a design matrix $X$ defined by a fifth-order polynomial and split your data set in training and test data."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "1b0cea0a",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "**b)**\n",
-    "Write thereafter (using either **scikit-learn** or your matrix inversion code using for example **numpy**)\n",
-    "and perform an ordinary least squares fitting and compute the mean squared error for the training data and the test data. These calculations should apply to a model given by a fifth-order polynomial."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "5194ba08",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "**c)**\n",
-    "Add now a model which allows you to make polynomials up to degree $15$.  Perform a standard OLS fitting of the training data and compute the MSE for the training and test data and plot both test and training data MSE as functions of the polynomial degree. Compare what you see with Figure 2.11 of Hastie et al. Comment your results. For which polynomial degree do you find an optimal MSE (smallest value)?"
-   ]
-  }
- ],
- "metadata": {},
- "nbformat": 4,
- "nbformat_minor": 5
-}
diff --git a/doc/LectureNotes/exercisesweek35.ipynb b/doc/LectureNotes/exercisesweek35.ipynb
deleted file mode 100644
index bc9ae7211..000000000
--- a/doc/LectureNotes/exercisesweek35.ipynb
+++ /dev/null
@@ -1,400 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "3080bd6e",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
-    "doconce format html exercisesweek35.do.txt  -->\n",
-    "<!-- dom:TITLE: Exercises week 35 -->"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "2121a646",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "# Exercises week 35\n",
-    "**August 26-30, 2024**\n",
-    "\n",
-    "Date: **Deadline is Friday August 30 at midnight**"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "5fdd1312",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "## Exercise 1: Analytical exercises\n",
-    "\n",
-    "In this exercise we derive the expressions for various derivatives of\n",
-    "products of vectors and matrices. Such derivatives are central to the\n",
-    "optimization of various cost functions. Although we will often use\n",
-    "automatic differentiation in actual calculations, to be able to have\n",
-    "analytical expressions is extremely helpful in case we have simpler\n",
-    "derivatives as well as when we analyze various properties (like second\n",
-    "derivatives) of the chosen cost functions.  Vectors are always written\n",
-    "as boldfaced lower case letters and matrices as upper case boldfaced\n",
-    "letters. You will find useful the notes from week 35 on derivatives of vectors and matrices.\n",
-    "See also the textbook of Faisal at al, chapter 5 and in particular sections 5.3-5.5 at <https://github.com/CompPhysics/MachineLearning/blob/master/doc/Textbooks/MathMLbook.pdf>\n",
-    "\n",
-    "Show that"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "5bd583f7",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\frac{\\partial (\\boldsymbol{a}^T\\boldsymbol{x})}{\\partial \\boldsymbol{x}} = \\boldsymbol{a}^T,\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "74d2af02",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "and"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "cc982163",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\frac{\\partial (\\boldsymbol{a}^T\\boldsymbol{A}\\boldsymbol{a})}{\\partial \\boldsymbol{a}} = \\boldsymbol{a}^T(\\boldsymbol{A}+\\boldsymbol{A}^T),\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "2f2c4da8",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "and"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "75abd105",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\frac{\\partial \\left(\\boldsymbol{x}-\\boldsymbol{A}\\boldsymbol{s}\\right)^T\\left(\\boldsymbol{x}-\\boldsymbol{A}\\boldsymbol{s}\\right)}{\\partial \\boldsymbol{s}} = -2\\left(\\boldsymbol{x}-\\boldsymbol{A}\\boldsymbol{s}\\right)^T\\boldsymbol{A},\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "ed48f425",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "and finally find the second derivative of this function with respect to the vector $\\boldsymbol{s}$. If we replace the vector $\\boldsymbol{s}$ with the unknown parameters $\\boldsymbol{\\beta}$ used to define the ordinary least squares method, we end up with the equations that determine these parameters. The matrix $\\boldsymbol{A}$ is then the design matrix $\\boldsymbol{X}$ and $\\boldsymbol{x}$ here has to be replaced with the outputs $\\boldsymbol{y}$.\n",
-    "\n",
-    "The second derivative of the mean squared error is then proportional to the so-called Hessian matrix $\\boldsymbol{H}=\\boldsymbol{X}^T\\boldsymbol{X}$.\n",
-    "\n",
-    "**Hint**: In these exercises it is always useful to write out with summation indices the various quantities. Take also a look at the weekly slides from week 35 and the various examples included there.\n",
-    "\n",
-    "As an example, consider the function"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "329edf30",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "f(\\boldsymbol{x}) =\\boldsymbol{A}\\boldsymbol{x},\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "01f1090f",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "which reads for a specific component $f_i$ (we define the matrix $\\boldsymbol{A}$ to have dimension $n\\times n$ and the vector $\\boldsymbol{x}$ to have length $n$)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "8c71ed48",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "f_i =\\sum_{j=0}^{n-1}a_{ij}x_j,\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "4e8012af",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "which leads to"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "a1e0c123",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\frac{\\partial f_i}{\\partial x_j}= a_{ij},\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "22a65aed",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "and written out in terms of the vector $\\boldsymbol{x}$ we have"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "e56f21e8",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\frac{\\partial f(\\boldsymbol{x})}{\\partial \\boldsymbol{x}}= \\boldsymbol{A}.\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "48394062",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "## Exercise 2: making your own data and exploring scikit-learn\n",
-    "\n",
-    "We will generate our own dataset for a function $y(x)$ where $x \\in\n",
-    "[0,1]$ and defined by random numbers computed with the uniform\n",
-    "distribution. The function $y$ is a quadratic polynomial in $x$ with\n",
-    "added stochastic noise according to the normal distribution $\\cal\n",
-    "{N}(0,1)$.  The following simple Python instructions define our $x$\n",
-    "and $y$ values (with 100 data points)."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "id": "bb67a97a",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
-   "outputs": [],
-   "source": [
-    "import numpy as np\n",
-    "x = np.random.rand(100,1)\n",
-    "y = 2.0+5*x*x+0.1*np.random.randn(100,1)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "d60db901",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "1. Write your own code (following the examples under the [regression notes](https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/chapter1.html)) for computing the parametrization of the data set fitting a second-order polynomial. \n",
-    "\n",
-    "2. Use thereafter **scikit-learn** (see again the examples in the slides for week 35) and compare with your own code. Note here that **scikit-learn** does not include, by default, the intercept. See the discussions on scaling your data in the slides for this week. This type of problems appear in particular if we fit a polynomial with an intercept. \n",
-    "\n",
-    "3. Using scikit-learn, compute also the mean squared error, a risk metric corresponding to the expected value of the squared (quadratic) error defined as"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "274328a7",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "MSE(\\boldsymbol{y},\\boldsymbol{\\tilde{y}}) = \\frac{1}{n}\n",
-    "\\sum_{i=0}^{n-1}(y_i-\\tilde{y}_i)^2,\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "59dce0f1",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "and the $R^2$ score function.\n",
-    "If $\\tilde{\\boldsymbol{y}}_i$ is the predicted value of the $i-th$ sample and $y_i$ is the corresponding true value, then the score $R^2$ is defined as"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "1e1f8911",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "R^2(\\boldsymbol{y}, \\tilde{\\boldsymbol{y}}) = 1 - \\frac{\\sum_{i=0}^{n - 1} (y_i - \\tilde{y}_i)^2}{\\sum_{i=0}^{n - 1} (y_i - \\bar{y})^2},\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "ef31124f",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "where we have defined the mean value  of $\\boldsymbol{y}$ as"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "46ac2123",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\bar{y} =  \\frac{1}{n} \\sum_{i=0}^{n - 1} y_i.\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "edf840af",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "You can use the functionality included in scikit-learn. If you feel for it, you can use your own program and define functions which compute the above two functions. \n",
-    "Discuss the meaning of these results. Try also to vary the coefficient in front of the added stochastic noise term and discuss the quality of the fits."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "31059187",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "## Exercise 3: Split data in test and training data\n",
-    "\n",
-    "In this exercise we want you to to compute the MSE for the training\n",
-    "data and the test data as function of the complexity of a polynomial,\n",
-    "that is the degree of a given polynomial.  \n",
-    "\n",
-    "The aim is to reproduce Figure 2.11 of [Hastie et al](https://github.com/CompPhysics/MLErasmus/blob/master/doc/Textbooks/elementsstat.pdf).\n",
-    "Feel free to read the discussions leading to figure 2.11 of Hastie et al. \n",
-    "\n",
-    "Our data is defined by $x\\in [-3,3]$ with a total of for example $n=100$ data points. You should try to vary the number of data points $n$ in your analysis."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "id": "f7382860",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
-   "outputs": [],
-   "source": [
-    "np.random.seed()\n",
-    "n = 100\n",
-    "# Make data set.\n",
-    "x = np.linspace(-3, 3, n).reshape(-1, 1)\n",
-    "y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "a10412bd",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "where $y$ is the function we want to fit with a given polynomial."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "8a9f0b89",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "**a)**\n",
-    "Write a first code which sets up a design matrix $X$ defined by a fifth-order polynomial and split your data set in training and test data."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "df5c7f67",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "**b)**\n",
-    "Write thereafter (using either **scikit-learn** or your matrix inversion code using for example **numpy**)\n",
-    "and perform an ordinary least squares fitting and compute the mean squared error for the training data and the test data. These calculations should apply to a model given by a fifth-order polynomial.  If you compare your own code with _scikit_learn_, not that the latter does not include by default the intercept. See the discussions on scaling your data in the slides for this week."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "e9b0fa39",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "**c)**\n",
-    "Add now a model which allows you to make polynomials up to degree $15$.  Perform a standard OLS fitting of the training data and compute the MSE for the training and test data and plot both test and training data MSE as functions of the polynomial degree. Compare what you see with Figure 2.11 of Hastie et al. Comment your results. For which polynomial degree do you find an optimal MSE (smallest value)?"
-   ]
-  }
- ],
- "metadata": {},
- "nbformat": 4,
- "nbformat_minor": 5
-}
diff --git a/doc/LectureNotes/exercisesweek36.ipynb b/doc/LectureNotes/exercisesweek36.ipynb
deleted file mode 100644
index 9bcf939ab..000000000
--- a/doc/LectureNotes/exercisesweek36.ipynb
+++ /dev/null
@@ -1,372 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "ba6ac392",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
-    "doconce format html exercisesweek36.do.txt  -->\n",
-    "<!-- dom:TITLE: Exercises week 36 -->"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "b6e88f71",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "# Exercises week 36\n",
-    "**September 2-6, 2024**\n",
-    "\n",
-    "Date: **Deadline is Friday September 6 at midnight**"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "bdcd6bb3",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "## Overarching aims of the exercises this week\n",
-    "\n",
-    "This set of exercises form an important part of the first project. The\n",
-    "analytical exercises deal with the material covered last week on the\n",
-    "mathematical interpretations of ordinary least squares and of Ridge\n",
-    "regression. The numerical exercises can be seen as a continuation of\n",
-    "exercise 3 from week 35, with the inclusion of Ridge regression. This\n",
-    "material enters also the discussions of the first project."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "06e77343",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "## Exercise 1: Analytical exercises\n",
-    "\n",
-    "The aim here is to derive the expression for the optimal parameters\n",
-    "using Ridge regression. Furthermore, using the singular value\n",
-    "decomposition, we will analyze the difference between the ordinary\n",
-    "least squares approach and Ridge regression.\n",
-    "\n",
-    "The expression for the standard Mean Squared Error (MSE) which we used to define our cost function and the equations for the ordinary least squares (OLS) method, was given by the\n",
-    "optimization problem"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "5025274e",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "{\\displaystyle \\min_{\\boldsymbol{\\beta}\\in {\\mathbb{R}}^{p}}}\\frac{1}{n}\\left\\{\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)\\right\\}.\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "900798b7",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "which we can also write as"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "c3916302",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "{\\displaystyle \\min_{\\boldsymbol{\\beta}\\in\n",
-    "{\\mathbb{R}}^{p}}}\\frac{1}{n}\\sum_{i=0}^{n-1}\\left(y_i-\\tilde{y}_i\\right)^2=\\frac{1}{n}\\vert\\vert \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\vert\\vert_2^2,\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "52a3d66c",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "where we have used the definition of  a norm-2 vector, that is"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "e537f20e",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\vert\\vert \\boldsymbol{x}\\vert\\vert_2 = \\sqrt{\\sum_i x_i^2}.\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "71e5f3a3",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "By minimizing the above equation with respect to the parameters\n",
-    "$\\boldsymbol{\\beta}$ we could then obtain an analytical expression for the\n",
-    "parameters $\\boldsymbol{\\beta}$.\n",
-    "\n",
-    "We can add a regularization parameter $\\lambda$ by\n",
-    "defining a new cost function to be optimized, that is"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "26fe46d4",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "{\\displaystyle \\min_{\\boldsymbol{\\beta}\\in\n",
-    "{\\mathbb{R}}^{p}}}\\frac{1}{n}\\vert\\vert \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\vert\\vert_2^2+\\lambda\\vert\\vert \\boldsymbol{\\beta}\\vert\\vert_2^2\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "13bacbe7",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "which leads to the Ridge regression minimization problem. One can require as part of the optimization problem \n",
-    "that $\\vert\\vert \\boldsymbol{\\beta}\\vert\\vert_2^2\\le t$, where $t$ is\n",
-    "a finite number larger than zero. We will not implement that here."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "3ca7c852",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "### a) Expression for Ridge regression\n",
-    "\n",
-    "Show that the optimal parameters"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "c6c84292",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\hat{\\boldsymbol{\\beta}}_{\\mathrm{Ridge}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}+\\lambda\\boldsymbol{I}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y},\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "15007b26",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "with $\\boldsymbol{I}$ being a $p\\times p$ identity matrix.\n",
-    "\n",
-    "The ordinary least squares result is"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "58f20595",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\hat{\\boldsymbol{\\beta}}_{\\mathrm{OLS}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y},\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "aab6662a",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "### b) The singular value decomposition\n",
-    "\n",
-    "Here we will use the singular value decomposition of an $n\\times p$ matrix $\\boldsymbol{X}$ (our design matrix)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "50358368",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\boldsymbol{X}=\\boldsymbol{U}\\boldsymbol{\\Sigma}\\boldsymbol{V}^T,\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "a87efdb8",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "to study properties of Ridge regression and ordinary least squares regression.\n",
-    "Here $\\boldsymbol{U}$ and $\\boldsymbol{V}$ are orthogonal matrices of dimensions\n",
-    "$n\\times n$ and $p\\times p$, respectively, and $\\boldsymbol{\\Sigma}$ is an\n",
-    "$n\\times p$ matrix which contains the singular values only. This material was discussed during the lectures of week 35.\n",
-    "\n",
-    "Show that you can write the \n",
-    "OLS solutions in terms of the eigenvectors (the columns) of the orthogonal matrix  $\\boldsymbol{U}$ as"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "23d5e69c",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\tilde{\\boldsymbol{y}}_{\\mathrm{OLS}}=\\boldsymbol{X}\\boldsymbol{\\beta}  = \\sum_{j=0}^{p-1}\\boldsymbol{u}_j\\boldsymbol{u}_j^T\\boldsymbol{y}.\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "f1086e51",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "For Ridge regression, show that the corresponding equation is"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "8cad7d30",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\tilde{\\boldsymbol{y}}_{\\mathrm{Ridge}}=\\boldsymbol{X}\\boldsymbol{\\beta}_{\\mathrm{Ridge}} = \\boldsymbol{U\\Sigma V^T}\\left(\\boldsymbol{V}\\boldsymbol{\\Sigma}^2\\boldsymbol{V}^T+\\lambda\\boldsymbol{I} \\right)^{-1}(\\boldsymbol{U\\Sigma V^T})^T\\boldsymbol{y}=\\sum_{j=0}^{p-1}\\boldsymbol{u}_j\\boldsymbol{u}_j^T\\frac{\\sigma_j^2}{\\sigma_j^2+\\lambda}\\boldsymbol{y},\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "a7e167a0",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "with the vectors $\\boldsymbol{u}_j$ being the columns of $\\boldsymbol{U}$ from the SVD of the matrix $\\boldsymbol{X}$. \n",
-    "\n",
-    "Give an interpretation of the results.  [Section 3.4 of Hastie et al's textbook gives a good discussion of the above results](https://link.springer.com/book/10.1007/978-0-387-84858-7)."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "a7482960",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "## Exercise 2: Adding Ridge Regression\n",
-    "\n",
-    "This exercise is a continuation of exercise 3 from week 35, see <https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/exercisesweek35.html>. We will use the same function to\n",
-    "generate our data set, still staying with a simple function $y(x)$\n",
-    "which we want to fit using linear regression, but now extending the\n",
-    "analysis to include the Ridge regression method.\n",
-    "\n",
-    "In this exercise you need to include the same elements from last week, that is\n",
-    "1. scale your data by subtracting the mean value from each column in the design matrix.\n",
-    "\n",
-    "2. perform a split of the data in a training set and a test set.\n",
-    "\n",
-    "The addition to the analysis this time is the introduction of the hyperparameter $\\lambda$ when introducing Ridge regression.\n",
-    "\n",
-    "Extend the code from exercise 3 from [week 35](https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/exercisesweek35.html) to include Ridge regression with the hyperparameter $\\lambda$. The optimal parameters $\\hat{\\beta}$ for Ridge regression can be obtained by matrix inversion in a similar way as done for ordinary least squares. You need to add to your code the following equations"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "abcb7cf9",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\hat{\\boldsymbol{\\beta}}_{\\mathrm{Ridge}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}+\\lambda\\boldsymbol{I}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y}.\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "bb76fdeb",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "The ordinary least squares result you encoded last week is given by"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "034da514",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\hat{\\boldsymbol{\\beta}}_{\\mathrm{OLS}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y},\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "8b5613a0",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "Use these results to compute the mean squared error for ordinary least\n",
-    "squares and Ridge regression first for a polynomial of degree five\n",
-    "with $n=100$ data points and five selected values of\n",
-    "$\\lambda=[0.0001,0.001, 0.01,0.1,1.0]$. Compute thereafter the mean\n",
-    "squared error for the same values of $\\lambda$ for polynomials of degree ten\n",
-    "and $15$. Discuss your results for the training MSE and test MSE with\n",
-    "Ridge regression and ordinary least squares."
-   ]
-  }
- ],
- "metadata": {},
- "nbformat": 4,
- "nbformat_minor": 5
-}
diff --git a/doc/LectureNotes/exercisesweek37.ipynb b/doc/LectureNotes/exercisesweek37.ipynb
deleted file mode 100644
index 5528002c1..000000000
--- a/doc/LectureNotes/exercisesweek37.ipynb
+++ /dev/null
@@ -1,268 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "2a3463de",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
-    "doconce format html exercisesweek37.do.txt  -->\n",
-    "<!-- dom:TITLE: Exercises week 37 -->"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "442e0844",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "# Exercises week 37\n",
-    "**September 9-13, 2024**\n",
-    "\n",
-    "Date: **Deadline is Friday September 13 at midnight**"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "0c0df373",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "## Overarching aims of the exercises this week\n",
-    "\n",
-    "This exercise deals with various mean values and variances in linear\n",
-    "regression method (here it may be useful to look up chapter 3,\n",
-    "equation (3.8) of [Trevor Hastie, Robert Tibshirani, Jerome\n",
-    "H. Friedman, The Elements of Statistical Learning,\n",
-    "Springer](https://www.springer.com/gp/book/9780387848570)). The\n",
-    "exercise is also a part of project 1 and can be reused in the theory\n",
-    "part of the project.\n",
-    "\n",
-    "For more discussions on Ridge regression and calculation of\n",
-    "expectation values, [Wessel van\n",
-    "Wieringen's](https://arxiv.org/abs/1509.09169) article is highly\n",
-    "recommended.\n",
-    "\n",
-    "The assumption we have made is that there exists a continuous function\n",
-    "$f(\\boldsymbol{x})$ and a normal distributed error $\\boldsymbol{\\varepsilon}\\sim N(0,\n",
-    "\\sigma^2)$ which describes our data"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "a1ac8666",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\boldsymbol{y} = f(\\boldsymbol{x})+\\boldsymbol{\\varepsilon}\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "ee6ce4be",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "We then approximate this function $f(\\boldsymbol{x})$ with our model $\\boldsymbol{\\tilde{y}}$ from the solution of the linear regression equations (ordinary least squares OLS), that is our\n",
-    "function $f$ is approximated by $\\boldsymbol{\\tilde{y}}$ where we minimized  $(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2$, with"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "2d50b2e4",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\boldsymbol{\\tilde{y}} = \\boldsymbol{X}\\boldsymbol{\\beta}.\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "3645cccd",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "The matrix $\\boldsymbol{X}$ is the so-called design or feature matrix."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "6a378434",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "## Exercise 1: Expectation values for ordinary least squares expressions\n",
-    "\n",
-    "Show that  the expectation value of $\\boldsymbol{y}$ for a given element $i$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "ce9b87a9",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\mathbb{E}(y_i)  =\\sum_{j}x_{ij} \\beta_j=\\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta},\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "a6435e39",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "and that\n",
-    "its variance is"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "75d89e64",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\mbox{Var}(y_i)  = \\sigma^2.\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "b5aef4d3",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "Hence, $y_i \\sim N( \\mathbf{X}_{i, \\ast} \\, \\boldsymbol{\\beta}, \\sigma^2)$, that is $\\boldsymbol{y}$ follows a normal distribution with \n",
-    "mean value $\\boldsymbol{X}\\boldsymbol{\\beta}$ and variance $\\sigma^2$.\n",
-    "\n",
-    "With the OLS expressions for the optimal parameters $\\boldsymbol{\\hat{\\beta}}$ show that"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "17012dce",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\mathbb{E}(\\boldsymbol{\\hat{\\beta}}) = \\boldsymbol{\\beta}.\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "4da3a821",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "Show finally that the variance of $\\boldsymbol{\\boldsymbol{\\beta}}$ is"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "0ba77c7b",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\mbox{Var}(\\boldsymbol{\\hat{\\beta}}) = \\sigma^2 \\, (\\mathbf{X}^{T} \\mathbf{X})^{-1}.\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "5c66de05",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "We can use the last expression when we define a [so-called confidence interval](https://en.wikipedia.org/wiki/Confidence_interval) for the parameters $\\beta$. \n",
-    "A given parameter $\\beta_j$ is given by the diagonal matrix element of the above matrix."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "43b92138",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "## Exercise 2: Expectation values for Ridge regression\n",
-    "\n",
-    "Show that"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "afea98ca",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\mathbb{E} \\big[ \\hat{\\boldsymbol{\\beta}}^{\\mathrm{Ridge}} \\big]=(\\mathbf{X}^{T} \\mathbf{X} + \\lambda \\mathbf{I}_{pp})^{-1} (\\mathbf{X}^{\\top} \\mathbf{X})\\boldsymbol{\\beta}.\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "13d7fd96",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "We see clearly that\n",
-    "$\\mathbb{E} \\big[ \\hat{\\boldsymbol{\\beta}}^{\\mathrm{Ridge}} \\big] \\not= \\mathbb{E} \\big[\\hat{\\boldsymbol{\\beta}}^{\\mathrm{OLS}}\\big ]$ for any $\\lambda > 0$.\n",
-    "\n",
-    "Show also that the variance is"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "77d1d47e",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\mbox{Var}[\\hat{\\boldsymbol{\\beta}}^{\\mathrm{Ridge}}]=\\sigma^2[  \\mathbf{X}^{T} \\mathbf{X} + \\lambda \\mathbf{I} ]^{-1}  \\mathbf{X}^{T}\\mathbf{X} \\{ [  \\mathbf{X}^{\\top} \\mathbf{X} + \\lambda \\mathbf{I} ]^{-1}\\}^{T},\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "2fa6bdfd",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "and it is easy to see that if the parameter $\\lambda$ goes to infinity then the variance of the Ridge parameters $\\boldsymbol{\\beta}$ goes to zero."
-   ]
-  }
- ],
- "metadata": {},
- "nbformat": 4,
- "nbformat_minor": 5
-}
diff --git a/doc/LectureNotes/exercisesweek38.ipynb b/doc/LectureNotes/exercisesweek38.ipynb
deleted file mode 100644
index eac507767..000000000
--- a/doc/LectureNotes/exercisesweek38.ipynb
+++ /dev/null
@@ -1,181 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "f9bcf943",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
-    "doconce format html exercisesweek38.do.txt  -->\n",
-    "<!-- dom:TITLE: Exercises week 38 -->"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "ead6d6d6",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "# Exercises week 38\n",
-    "**September 16-20, 2024**\n",
-    "\n",
-    "Date: **Deadline is Friday September 20 at midnight**"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "b6ce9344",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "## Overarching aims of the exercises this week\n",
-    "\n",
-    "The aim of the exercises this week is to derive the equations for the bias-variance tradeoff to be used in project 1 as well as testing this for a simpler function using the bootstrap method. The exercises here can be reused in project 1 as well.\n",
-    "\n",
-    "Consider a\n",
-    "dataset $\\mathcal{L}$ consisting of the data\n",
-    "$\\mathbf{X}_\\mathcal{L}=\\{(y_j, \\boldsymbol{x}_j), j=0\\ldots n-1\\}$.\n",
-    "\n",
-    "We assume that the true data is generated from a noisy model"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "29371f21",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\boldsymbol{y}=f(\\boldsymbol{x}) + \\boldsymbol{\\epsilon}.\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "36a8765e",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "Here $\\epsilon$ is normally distributed with mean zero and standard\n",
-    "deviation $\\sigma^2$.\n",
-    "\n",
-    "In our derivation of the ordinary least squares method we defined \n",
-    "an approximation to the function $f$ in terms of the parameters\n",
-    "$\\boldsymbol{\\beta}$ and the design matrix $\\boldsymbol{X}$ which embody our model,\n",
-    "that is $\\boldsymbol{\\tilde{y}}=\\boldsymbol{X}\\boldsymbol{\\beta}$.\n",
-    "\n",
-    "The parameters $\\boldsymbol{\\beta}$ are in turn found by optimizing the mean\n",
-    "squared error via the so-called cost function"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "68dd52df",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "C(\\boldsymbol{X},\\boldsymbol{\\beta}) =\\frac{1}{n}\\sum_{i=0}^{n-1}(y_i-\\tilde{y}_i)^2=\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right].\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "68a07606",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "Here the expected value $\\mathbb{E}$ is the sample value. \n",
-    "\n",
-    "Show that you can rewrite  this in terms of a term which contains the variance of the model itself (the so-called variance term), a\n",
-    "term which measures the deviation from the true data and the mean value of the model (the bias term) and finally the variance of the noise.\n",
-    "That is, show that"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "fbbb3fd7",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=\\mathrm{Bias}[\\tilde{y}]+\\mathrm{var}[\\tilde{y}]+\\sigma^2,\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "5ecc0f22",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "with"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "8c17bd0a",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\mathrm{Bias}[\\tilde{y}]=\\mathbb{E}\\left[\\left(\\boldsymbol{y}-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right]\\right)^2\\right],\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "55ab4f0c",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "and"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "98f93c68",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "$$\n",
-    "\\mathrm{var}[\\tilde{y}]=\\mathbb{E}\\left[\\left(\\tilde{\\boldsymbol{y}}-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right]\\right)^2\\right]=\\frac{1}{n}\\sum_i(\\tilde{y}_i-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2.\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "d6d099d4",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "Explain what the terms mean and discuss their interpretations.\n",
-    "\n",
-    "Perform then a bias-variance analysis of a simple one-dimensional (or other models of your choice)  function by\n",
-    "studying the MSE value as function of the complexity of your model. Use ordinary least squares only.\n",
-    "\n",
-    "Discuss the bias and variance trade-off as function\n",
-    "of your model complexity (the degree of the polynomial) and the number\n",
-    "of data points, and possibly also your training and test data using the **bootstrap** resampling method.\n",
-    "You can follow the code example in the jupyter-book at <https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/chapter3.html#the-bias-variance-tradeoff>."
-   ]
-  }
- ],
- "metadata": {},
- "nbformat": 4,
- "nbformat_minor": 5
-}
diff --git a/doc/LectureNotes/exercisesweek39.ipynb b/doc/LectureNotes/exercisesweek39.ipynb
deleted file mode 100644
index 32af98ace..000000000
--- a/doc/LectureNotes/exercisesweek39.ipynb
+++ /dev/null
@@ -1,59 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "12bcd5bb",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
-    "doconce format html exercisesweek39.do.txt  -->\n",
-    "<!-- dom:TITLE: Exercises week 39 -->"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "cd59f741",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "# Exercises week 39\n",
-    "**September 23-27, 2024**\n",
-    "\n",
-    "Date: **Deadline is Friday September 27 at midnight**"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "bdf13ce1",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "## Overarching aims of the exercises this week\n",
-    "\n",
-    "The aim of the exercises this week is to aid you in getting started\n",
-    "with writing the report. This will be discussed during the lab\n",
-    "sessions as well. \n",
-    "\n",
-    "A general guideline can be found at <https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md>.\n",
-    "\n",
-    "Similarly, an example of an earlier project can be found at <https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/ReportSample.pdf>\n",
-    "\n",
-    "Your task this week is to\n",
-    "1. Write an abstract for your project\n",
-    "\n",
-    "2. Write an introduction\n",
-    "\n",
-    "3. Include references\n",
-    "\n",
-    "A short feedback to the this exercise will be available before the project deadline. And you can reuse these elements in your final report."
-   ]
-  }
- ],
- "metadata": {},
- "nbformat": 4,
- "nbformat_minor": 5
-}
diff --git a/doc/LectureNotes/exercisesweek41.ipynb b/doc/LectureNotes/exercisesweek41.ipynb
deleted file mode 100644
index 739fb1439..000000000
--- a/doc/LectureNotes/exercisesweek41.ipynb
+++ /dev/null
@@ -1,1066 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "b18bcd06",
-   "metadata": {},
-   "source": [
-    "<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
-    "doconce format html exercisesweek41.do.txt  -->\n",
-    "<!-- dom:TITLE: Exercises week 41 -->"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "7542d6aa",
-   "metadata": {},
-   "source": [
-    "# Exercises week 41\n",
-    "**October 4-11, 2024**\n",
-    "\n",
-    "Date: **Deadline is Friday October 11 at midnight**"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "80943a15",
-   "metadata": {},
-   "source": [
-    "# Overarching aims of the exercises this week\n",
-    "\n",
-    "The aim of the exercises this week is to get started with implementing\n",
-    "gradient methods of relevance for project 2. This exercise will also\n",
-    "be continued next week with the addition of automatic differentation.\n",
-    "Everything you develop here will be used in project 2.\n",
-    "\n",
-    "In order to get started, we will now replace in our standard ordinary\n",
-    "least squares (OLS) and Ridge regression codes (from project 1) the\n",
-    "matrix inversion algorithm with our own gradient descent (GD) and SGD\n",
-    "codes.  You can use the Franke function or the terrain data from\n",
-    "project 1. **However, we recommend using a simpler function like**\n",
-    "$f(x)=a_0+a_1x+a_2x^2$ or higher-order one-dimensional polynomials.\n",
-    "You can obviously test your final codes against for example the Franke\n",
-    "function. Automatic differentiation will be discussed next week.\n",
-    "\n",
-    "You should include in your analysis of the GD and SGD codes the following elements\n",
-    "1. A plain gradient descent with a fixed learning rate (you will need to tune it) using the analytical expression of the gradients\n",
-    "\n",
-    "2. Add momentum to the plain GD code and compare convergence with a fixed learning rate (you may need to tune the learning rate), again using the analytical expression of the gradients.\n",
-    "\n",
-    "3. Repeat these steps for stochastic gradient descent with mini batches and a given number of epochs. Use a tunable learning rate as discussed in the lectures from week 39. Discuss the results as functions of the various parameters (size of batches, number of epochs etc)\n",
-    "\n",
-    "4. Implement the Adagrad method in order to tune the learning rate. Do this with and without momentum for plain gradient descent and SGD.\n",
-    "\n",
-    "5. Add RMSprop and Adam to your library of methods for tuning the learning rate.\n",
-    "\n",
-    "The lecture notes from weeks 39 and 40 contain more information and code examples. Feel free to use these examples.\n",
-    "\n",
-    "In summary, you should \n",
-    "perform an analysis of the results for OLS and Ridge regression as\n",
-    "function of the chosen learning rates, the number of mini-batches and\n",
-    "epochs as well as algorithm for scaling the learning rate. You can\n",
-    "also compare your own results with those that can be obtained using\n",
-    "for example **Scikit-Learn**'s various SGD options.  Discuss your\n",
-    "results. For Ridge regression you need now to study the results as functions of  the hyper-parameter $\\lambda$ and \n",
-    "the learning rate $\\eta$.  Discuss your results.\n",
-    "\n",
-    "You will need your SGD code for the setup of the Neural Network and\n",
-    "Logistic Regression codes. You will find the Python [Seaborn\n",
-    "package](https://seaborn.pydata.org/generated/seaborn.heatmap.html)\n",
-    "useful when plotting the results as function of the learning rate\n",
-    "$\\eta$ and the hyper-parameter $\\lambda$ when you use Ridge\n",
-    "regression.\n",
-    "\n",
-    "We recommend reading chapter 8 on optimization from the textbook of [Goodfellow, Bengio and Courville](https://www.deeplearningbook.org/). This chapter contains many useful insights and discussions on the optimization part of machine learning."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "2095d197",
-   "metadata": {},
-   "source": [
-    "# Code examples from week 39 and 40"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "f428decb",
-   "metadata": {},
-   "source": [
-    "## Code with a Number of Minibatches which varies, analytical gradient\n",
-    "\n",
-    "In the code here we vary the number of mini-batches."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 1,
-   "id": "ba38d454",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "%matplotlib inline\n",
-    "\n",
-    "# Importing various packages\n",
-    "from math import exp, sqrt\n",
-    "from random import random, seed\n",
-    "import numpy as np\n",
-    "import matplotlib.pyplot as plt\n",
-    "\n",
-    "n = 100\n",
-    "x = 2*np.random.rand(n,1)\n",
-    "y = 4+3*x+np.random.randn(n,1)\n",
-    "\n",
-    "X = np.c_[np.ones((n,1)), x]\n",
-    "XT_X = X.T @ X\n",
-    "theta_linreg = np.linalg.inv(X.T @ X) @ (X.T @ y)\n",
-    "print(\"Own inversion\")\n",
-    "print(theta_linreg)\n",
-    "# Hessian matrix\n",
-    "H = (2.0/n)* XT_X\n",
-    "EigValues, EigVectors = np.linalg.eig(H)\n",
-    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
-    "\n",
-    "theta = np.random.randn(2,1)\n",
-    "eta = 1.0/np.max(EigValues)\n",
-    "Niterations = 1000\n",
-    "\n",
-    "#while (iter <= Ni... or test)\n",
-    "for iter in range(Niterations):\n",
-    "    gradients = 2.0/n*X.T @ ((X @ theta)-y)\n",
-    "    theta -= eta*gradients\n",
-    "print(\"theta from own gd\")\n",
-    "print(theta)\n",
-    "\n",
-    "xnew = np.array([[0],[2]])\n",
-    "Xnew = np.c_[np.ones((2,1)), xnew]\n",
-    "ypredict = Xnew.dot(theta)\n",
-    "ypredict2 = Xnew.dot(theta_linreg)\n",
-    "\n",
-    "n_epochs = 50\n",
-    "M = 5   #size of each minibatch\n",
-    "m = int(n/M) #number of minibatches\n",
-    "t0, t1 = 5, 50\n",
-    "\n",
-    "def learning_schedule(t):\n",
-    "    return t0/(t+t1)\n",
-    "\n",
-    "theta = np.random.randn(2,1)\n",
-    "\n",
-    "for epoch in range(n_epochs):\n",
-    "# Can you figure out a better way of setting up the contributions to each batch?\n",
-    "    for i in range(m):\n",
-    "        random_index = M*np.random.randint(m)\n",
-    "        xi = X[random_index:random_index+M]\n",
-    "        yi = y[random_index:random_index+M]\n",
-    "        gradients = (2.0/M)* xi.T @ ((xi @ theta)-yi)\n",
-    "        eta = learning_schedule(epoch*m+i)\n",
-    "        theta = theta - eta*gradients\n",
-    "print(\"theta from own sdg\")\n",
-    "print(theta)\n",
-    "\n",
-    "plt.plot(xnew, ypredict, \"r-\")\n",
-    "plt.plot(xnew, ypredict2, \"b-\")\n",
-    "plt.plot(x, y ,'ro')\n",
-    "plt.axis([0,2.0,0, 15.0])\n",
-    "plt.xlabel(r'$x$')\n",
-    "plt.ylabel(r'$y$')\n",
-    "plt.title(r'Random numbers ')\n",
-    "plt.show()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "de04b41a",
-   "metadata": {},
-   "source": [
-    "In the above code, we have use replacement in setting up the\n",
-    "mini-batches. The discussion\n",
-    "[here](https://sebastianraschka.com/faq/docs/sgd-methods.html) may be\n",
-    "useful."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "77fc1cca",
-   "metadata": {},
-   "source": [
-    "## Momentum based GD\n",
-    "\n",
-    "The stochastic gradient descent (SGD) is almost always used with a\n",
-    "*momentum* or inertia term that serves as a memory of the direction we\n",
-    "are moving in parameter space.  This is typically implemented as\n",
-    "follows"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "441d1f36",
-   "metadata": {},
-   "source": [
-    "$$\n",
-    "\\mathbf{v}_{t}=\\gamma \\mathbf{v}_{t-1}+\\eta_{t}\\nabla_\\theta E(\\boldsymbol{\\theta}_t) \\nonumber\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "47434945",
-   "metadata": {},
-   "source": [
-    "<!-- Equation labels as ordinary links -->\n",
-    "<div id=\"_auto1\"></div>\n",
-    "\n",
-    "$$\n",
-    "\\begin{equation} \n",
-    "\\boldsymbol{\\theta}_{t+1}= \\boldsymbol{\\theta}_t -\\mathbf{v}_{t},\n",
-    "\\label{_auto1} \\tag{1}\n",
-    "\\end{equation}\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "f3ea5060",
-   "metadata": {},
-   "source": [
-    "where we have introduced a momentum parameter $\\gamma$, with\n",
-    "$0\\le\\gamma\\le 1$, and for brevity we dropped the explicit notation to\n",
-    "indicate the gradient is to be taken over a different mini-batch at\n",
-    "each step. We call this algorithm gradient descent with momentum\n",
-    "(GDM). From these equations, it is clear that $\\mathbf{v}_t$ is a\n",
-    "running average of recently encountered gradients and\n",
-    "$(1-\\gamma)^{-1}$ sets the characteristic time scale for the memory\n",
-    "used in the averaging procedure. Consistent with this, when\n",
-    "$\\gamma=0$, this just reduces down to ordinary SGD as discussed\n",
-    "earlier. An equivalent way of writing the updates is"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "923628c8",
-   "metadata": {},
-   "source": [
-    "$$\n",
-    "\\Delta \\boldsymbol{\\theta}_{t+1} = \\gamma \\Delta \\boldsymbol{\\theta}_t -\\ \\eta_{t}\\nabla_\\theta E(\\boldsymbol{\\theta}_t),\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "5c94031c",
-   "metadata": {},
-   "source": [
-    "where we have defined $\\Delta \\boldsymbol{\\theta}_{t}= \\boldsymbol{\\theta}_t-\\boldsymbol{\\theta}_{t-1}$."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "f3f0e9c9",
-   "metadata": {},
-   "source": [
-    "## Algorithms and codes for Adagrad, RMSprop and Adam\n",
-    "\n",
-    "The algorithms we have implemented are well described in the text by [Goodfellow, Bengio and Courville, chapter 8](https://www.deeplearningbook.org/contents/optimization.html).\n",
-    "\n",
-    "The codes which implement these algorithms are discussed after our presentation of automatic differentiation."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "92253eff",
-   "metadata": {},
-   "source": [
-    "## Practical tips\n",
-    "\n",
-    "* **Randomize the data when making mini-batches**. It is always important to randomly shuffle the data when forming mini-batches. Otherwise, the gradient descent method can fit spurious correlations resulting from the order in which data is presented.\n",
-    "\n",
-    "* **Transform your inputs**. Learning becomes difficult when our landscape has a mixture of steep and flat directions. One simple trick for minimizing these situations is to standardize the data by subtracting the mean and normalizing the variance of input variables. Whenever possible, also decorrelate the inputs. To understand why this is helpful, consider the case of linear regression. It is easy to show that for the squared error cost function, the Hessian of the cost function is just the correlation matrix between the inputs. Thus, by standardizing the inputs, we are ensuring that the landscape looks homogeneous in all directions in parameter space. Since most deep networks can be viewed as linear transformations followed by a non-linearity at each layer, we expect this intuition to hold beyond the linear case.\n",
-    "\n",
-    "* **Monitor the out-of-sample performance.** Always monitor the performance of your model on a validation set (a small portion of the training data that is held out of the training process to serve as a proxy for the test set. If the validation error starts increasing, then the model is beginning to overfit. Terminate the learning process. This *early stopping* significantly improves performance in many settings.\n",
-    "\n",
-    "* **Adaptive optimization methods don't always have good generalization.** Recent studies have shown that adaptive methods such as ADAM, RMSPorp, and AdaGrad tend to have poor generalization compared to SGD or SGD with momentum, particularly in the high-dimensional limit (i.e. the number of parameters exceeds the number of data points). Although it is not clear at this stage why these methods perform so well in training deep neural networks, simpler procedures like properly-tuned SGD may work as well or better in these applications.\n",
-    "\n",
-    "Geron's text, see chapter 11, has several interesting discussions."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "08209015",
-   "metadata": {},
-   "source": [
-    "## Using Automatic differentation with OLS\n",
-    "\n",
-    "We conclude the part on optmization by showing how we can make codes\n",
-    "for linear regression and logistic regression using **autograd**. The\n",
-    "first example shows results with ordinary leats squares."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 2,
-   "id": "f1f7d4aa",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Using Autograd to calculate gradients for OLS\n",
-    "from random import random, seed\n",
-    "import numpy as np\n",
-    "import autograd.numpy as np\n",
-    "import matplotlib.pyplot as plt\n",
-    "from autograd import grad\n",
-    "\n",
-    "def CostOLS(beta):\n",
-    "    return (1.0/n)*np.sum((y-X @ beta)**2)\n",
-    "\n",
-    "n = 100\n",
-    "x = 2*np.random.rand(n,1)\n",
-    "y = 4+3*x+np.random.randn(n,1)\n",
-    "\n",
-    "X = np.c_[np.ones((n,1)), x]\n",
-    "XT_X = X.T @ X\n",
-    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
-    "print(\"Own inversion\")\n",
-    "print(theta_linreg)\n",
-    "# Hessian matrix\n",
-    "H = (2.0/n)* XT_X\n",
-    "EigValues, EigVectors = np.linalg.eig(H)\n",
-    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
-    "\n",
-    "theta = np.random.randn(2,1)\n",
-    "eta = 1.0/np.max(EigValues)\n",
-    "Niterations = 1000\n",
-    "# define the gradient\n",
-    "training_gradient = grad(CostOLS)\n",
-    "\n",
-    "for iter in range(Niterations):\n",
-    "    gradients = training_gradient(theta)\n",
-    "    theta -= eta*gradients\n",
-    "print(\"theta from own gd\")\n",
-    "print(theta)\n",
-    "\n",
-    "xnew = np.array([[0],[2]])\n",
-    "Xnew = np.c_[np.ones((2,1)), xnew]\n",
-    "ypredict = Xnew.dot(theta)\n",
-    "ypredict2 = Xnew.dot(theta_linreg)\n",
-    "\n",
-    "plt.plot(xnew, ypredict, \"r-\")\n",
-    "plt.plot(xnew, ypredict2, \"b-\")\n",
-    "plt.plot(x, y ,'ro')\n",
-    "plt.axis([0,2.0,0, 15.0])\n",
-    "plt.xlabel(r'$x$')\n",
-    "plt.ylabel(r'$y$')\n",
-    "plt.title(r'Random numbers ')\n",
-    "plt.show()"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "1bc83f33",
-   "metadata": {},
-   "source": [
-    "## Same code but now with momentum gradient descent"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "id": "dc2a3f65",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Using Autograd to calculate gradients for OLS\n",
-    "from random import random, seed\n",
-    "import numpy as np\n",
-    "import autograd.numpy as np\n",
-    "import matplotlib.pyplot as plt\n",
-    "from autograd import grad\n",
-    "\n",
-    "def CostOLS(beta):\n",
-    "    return (1.0/n)*np.sum((y-X @ beta)**2)\n",
-    "\n",
-    "n = 100\n",
-    "x = 2*np.random.rand(n,1)\n",
-    "y = 4+3*x#+np.random.randn(n,1)\n",
-    "\n",
-    "X = np.c_[np.ones((n,1)), x]\n",
-    "XT_X = X.T @ X\n",
-    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
-    "print(\"Own inversion\")\n",
-    "print(theta_linreg)\n",
-    "# Hessian matrix\n",
-    "H = (2.0/n)* XT_X\n",
-    "EigValues, EigVectors = np.linalg.eig(H)\n",
-    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
-    "\n",
-    "theta = np.random.randn(2,1)\n",
-    "eta = 1.0/np.max(EigValues)\n",
-    "Niterations = 30\n",
-    "\n",
-    "# define the gradient\n",
-    "training_gradient = grad(CostOLS)\n",
-    "\n",
-    "for iter in range(Niterations):\n",
-    "    gradients = training_gradient(theta)\n",
-    "    theta -= eta*gradients\n",
-    "    print(iter,gradients[0],gradients[1])\n",
-    "print(\"theta from own gd\")\n",
-    "print(theta)\n",
-    "\n",
-    "# Now improve with momentum gradient descent\n",
-    "change = 0.0\n",
-    "delta_momentum = 0.3\n",
-    "for iter in range(Niterations):\n",
-    "    # calculate gradient\n",
-    "    gradients = training_gradient(theta)\n",
-    "    # calculate update\n",
-    "    new_change = eta*gradients+delta_momentum*change\n",
-    "    # take a step\n",
-    "    theta -= new_change\n",
-    "    # save the change\n",
-    "    change = new_change\n",
-    "    print(iter,gradients[0],gradients[1])\n",
-    "print(\"theta from own gd wth momentum\")\n",
-    "print(theta)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "0ef007d0",
-   "metadata": {},
-   "source": [
-    "## But noen of these can compete with Newton's method"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 4,
-   "id": "0e498aa4",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Using Newton's method\n",
-    "from random import random, seed\n",
-    "import numpy as np\n",
-    "import autograd.numpy as np\n",
-    "import matplotlib.pyplot as plt\n",
-    "from autograd import grad\n",
-    "\n",
-    "def CostOLS(beta):\n",
-    "    return (1.0/n)*np.sum((y-X @ beta)**2)\n",
-    "\n",
-    "n = 100\n",
-    "x = 2*np.random.rand(n,1)\n",
-    "y = 4+3*x+np.random.randn(n,1)\n",
-    "\n",
-    "X = np.c_[np.ones((n,1)), x]\n",
-    "XT_X = X.T @ X\n",
-    "beta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
-    "print(\"Own inversion\")\n",
-    "print(beta_linreg)\n",
-    "# Hessian matrix\n",
-    "H = (2.0/n)* XT_X\n",
-    "# Note that here the Hessian does not depend on the parameters beta\n",
-    "invH = np.linalg.pinv(H)\n",
-    "EigValues, EigVectors = np.linalg.eig(H)\n",
-    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
-    "\n",
-    "beta = np.random.randn(2,1)\n",
-    "Niterations = 5\n",
-    "\n",
-    "# define the gradient\n",
-    "training_gradient = grad(CostOLS)\n",
-    "\n",
-    "for iter in range(Niterations):\n",
-    "    gradients = training_gradient(beta)\n",
-    "    beta -= invH @ gradients\n",
-    "    print(iter,gradients[0],gradients[1])\n",
-    "print(\"beta from own Newton code\")\n",
-    "print(beta)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "40292cf3",
-   "metadata": {},
-   "source": [
-    "## Including Stochastic Gradient Descent with Autograd\n",
-    "In this code we include the stochastic gradient descent approach discussed above. Note here that we specify which argument we are taking the derivative with respect to when using **autograd**."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "id": "fa819b9d",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Using Autograd to calculate gradients using SGD\n",
-    "# OLS example\n",
-    "from random import random, seed\n",
-    "import numpy as np\n",
-    "import autograd.numpy as np\n",
-    "import matplotlib.pyplot as plt\n",
-    "from autograd import grad\n",
-    "\n",
-    "# Note change from previous example\n",
-    "def CostOLS(y,X,theta):\n",
-    "    return np.sum((y-X @ theta)**2)\n",
-    "\n",
-    "n = 100\n",
-    "x = 2*np.random.rand(n,1)\n",
-    "y = 4+3*x+np.random.randn(n,1)\n",
-    "\n",
-    "X = np.c_[np.ones((n,1)), x]\n",
-    "XT_X = X.T @ X\n",
-    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
-    "print(\"Own inversion\")\n",
-    "print(theta_linreg)\n",
-    "# Hessian matrix\n",
-    "H = (2.0/n)* XT_X\n",
-    "EigValues, EigVectors = np.linalg.eig(H)\n",
-    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
-    "\n",
-    "theta = np.random.randn(2,1)\n",
-    "eta = 1.0/np.max(EigValues)\n",
-    "Niterations = 1000\n",
-    "\n",
-    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
-    "training_gradient = grad(CostOLS,2)\n",
-    "\n",
-    "for iter in range(Niterations):\n",
-    "    gradients = (1.0/n)*training_gradient(y, X, theta)\n",
-    "    theta -= eta*gradients\n",
-    "print(\"theta from own gd\")\n",
-    "print(theta)\n",
-    "\n",
-    "xnew = np.array([[0],[2]])\n",
-    "Xnew = np.c_[np.ones((2,1)), xnew]\n",
-    "ypredict = Xnew.dot(theta)\n",
-    "ypredict2 = Xnew.dot(theta_linreg)\n",
-    "\n",
-    "plt.plot(xnew, ypredict, \"r-\")\n",
-    "plt.plot(xnew, ypredict2, \"b-\")\n",
-    "plt.plot(x, y ,'ro')\n",
-    "plt.axis([0,2.0,0, 15.0])\n",
-    "plt.xlabel(r'$x$')\n",
-    "plt.ylabel(r'$y$')\n",
-    "plt.title(r'Random numbers ')\n",
-    "plt.show()\n",
-    "\n",
-    "n_epochs = 50\n",
-    "M = 5   #size of each minibatch\n",
-    "m = int(n/M) #number of minibatches\n",
-    "t0, t1 = 5, 50\n",
-    "def learning_schedule(t):\n",
-    "    return t0/(t+t1)\n",
-    "\n",
-    "theta = np.random.randn(2,1)\n",
-    "\n",
-    "for epoch in range(n_epochs):\n",
-    "# Can you figure out a better way of setting up the contributions to each batch?\n",
-    "    for i in range(m):\n",
-    "        random_index = M*np.random.randint(m)\n",
-    "        xi = X[random_index:random_index+M]\n",
-    "        yi = y[random_index:random_index+M]\n",
-    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
-    "        eta = learning_schedule(epoch*m+i)\n",
-    "        theta = theta - eta*gradients\n",
-    "print(\"theta from own sdg\")\n",
-    "print(theta)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "2ca466b4",
-   "metadata": {},
-   "source": [
-    "## Same code but now with momentum gradient descent"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 6,
-   "id": "0d44a49c",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Using Autograd to calculate gradients using SGD\n",
-    "# OLS example\n",
-    "from random import random, seed\n",
-    "import numpy as np\n",
-    "import autograd.numpy as np\n",
-    "import matplotlib.pyplot as plt\n",
-    "from autograd import grad\n",
-    "\n",
-    "# Note change from previous example\n",
-    "def CostOLS(y,X,theta):\n",
-    "    return np.sum((y-X @ theta)**2)\n",
-    "\n",
-    "n = 100\n",
-    "x = 2*np.random.rand(n,1)\n",
-    "y = 4+3*x+np.random.randn(n,1)\n",
-    "\n",
-    "X = np.c_[np.ones((n,1)), x]\n",
-    "XT_X = X.T @ X\n",
-    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
-    "print(\"Own inversion\")\n",
-    "print(theta_linreg)\n",
-    "# Hessian matrix\n",
-    "H = (2.0/n)* XT_X\n",
-    "EigValues, EigVectors = np.linalg.eig(H)\n",
-    "print(f\"Eigenvalues of Hessian Matrix:{EigValues}\")\n",
-    "\n",
-    "theta = np.random.randn(2,1)\n",
-    "eta = 1.0/np.max(EigValues)\n",
-    "Niterations = 100\n",
-    "\n",
-    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
-    "training_gradient = grad(CostOLS,2)\n",
-    "\n",
-    "for iter in range(Niterations):\n",
-    "    gradients = (1.0/n)*training_gradient(y, X, theta)\n",
-    "    theta -= eta*gradients\n",
-    "print(\"theta from own gd\")\n",
-    "print(theta)\n",
-    "\n",
-    "\n",
-    "n_epochs = 50\n",
-    "M = 5   #size of each minibatch\n",
-    "m = int(n/M) #number of minibatches\n",
-    "t0, t1 = 5, 50\n",
-    "def learning_schedule(t):\n",
-    "    return t0/(t+t1)\n",
-    "\n",
-    "theta = np.random.randn(2,1)\n",
-    "\n",
-    "change = 0.0\n",
-    "delta_momentum = 0.3\n",
-    "\n",
-    "for epoch in range(n_epochs):\n",
-    "    for i in range(m):\n",
-    "        random_index = M*np.random.randint(m)\n",
-    "        xi = X[random_index:random_index+M]\n",
-    "        yi = y[random_index:random_index+M]\n",
-    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
-    "        eta = learning_schedule(epoch*m+i)\n",
-    "        # calculate update\n",
-    "        new_change = eta*gradients+delta_momentum*change\n",
-    "        # take a step\n",
-    "        theta -= new_change\n",
-    "        # save the change\n",
-    "        change = new_change\n",
-    "print(\"theta from own sdg with momentum\")\n",
-    "print(theta)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "b82627f6",
-   "metadata": {},
-   "source": [
-    "## AdaGrad algorithm, taken from [Goodfellow et al](https://www.deeplearningbook.org/contents/optimization.html)\n",
-    "\n",
-    "<!-- dom:FIGURE: [figures/adagrad.png, width=600 frac=0.8] -->\n",
-    "<!-- begin figure -->\n",
-    "\n",
-    "<img src=\"figures/adagrad.png\" width=\"600\"><p style=\"font-size: 0.9em\"><i>Figure 1: </i></p>\n",
-    "<!-- end figure -->"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "00d3aff0",
-   "metadata": {},
-   "source": [
-    "## Similar (second order function now) problem but now with AdaGrad"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 7,
-   "id": "6b85aacc",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Using Autograd to calculate gradients using AdaGrad and Stochastic Gradient descent\n",
-    "# OLS example\n",
-    "from random import random, seed\n",
-    "import numpy as np\n",
-    "import autograd.numpy as np\n",
-    "import matplotlib.pyplot as plt\n",
-    "from autograd import grad\n",
-    "\n",
-    "# Note change from previous example\n",
-    "def CostOLS(y,X,theta):\n",
-    "    return np.sum((y-X @ theta)**2)\n",
-    "\n",
-    "n = 1000\n",
-    "x = np.random.rand(n,1)\n",
-    "y = 2.0+3*x +4*x*x\n",
-    "\n",
-    "X = np.c_[np.ones((n,1)), x, x*x]\n",
-    "XT_X = X.T @ X\n",
-    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
-    "print(\"Own inversion\")\n",
-    "print(theta_linreg)\n",
-    "\n",
-    "\n",
-    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
-    "training_gradient = grad(CostOLS,2)\n",
-    "# Define parameters for Stochastic Gradient Descent\n",
-    "n_epochs = 50\n",
-    "M = 5   #size of each minibatch\n",
-    "m = int(n/M) #number of minibatches\n",
-    "# Guess for unknown parameters theta\n",
-    "theta = np.random.randn(3,1)\n",
-    "\n",
-    "# Value for learning rate\n",
-    "eta = 0.01\n",
-    "# Including AdaGrad parameter to avoid possible division by zero\n",
-    "delta  = 1e-8\n",
-    "for epoch in range(n_epochs):\n",
-    "    Giter = 0.0\n",
-    "    for i in range(m):\n",
-    "        random_index = M*np.random.randint(m)\n",
-    "        xi = X[random_index:random_index+M]\n",
-    "        yi = y[random_index:random_index+M]\n",
-    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
-    "        Giter += gradients*gradients\n",
-    "        update = gradients*eta/(delta+np.sqrt(Giter))\n",
-    "        theta -= update\n",
-    "print(\"theta from own AdaGrad\")\n",
-    "print(theta)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "d8ddde38",
-   "metadata": {},
-   "source": [
-    "Running this code we note an almost perfect agreement with the results from matrix inversion."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "ff15b503",
-   "metadata": {},
-   "source": [
-    "## RMSProp algorithm, taken from [Goodfellow et al](https://www.deeplearningbook.org/contents/optimization.html)\n",
-    "\n",
-    "<!-- dom:FIGURE: [figures/rmsprop.png, width=600 frac=0.8] -->\n",
-    "<!-- begin figure -->\n",
-    "\n",
-    "<img src=\"figures/rmsprop.png\" width=\"600\"><p style=\"font-size: 0.9em\"><i>Figure 1: </i></p>\n",
-    "<!-- end figure -->"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "66f96d12",
-   "metadata": {},
-   "source": [
-    "## RMSprop for adaptive learning rate with Stochastic Gradient Descent"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 8,
-   "id": "888f1b4e",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Using Autograd to calculate gradients using RMSprop  and Stochastic Gradient descent\n",
-    "# OLS example\n",
-    "from random import random, seed\n",
-    "import numpy as np\n",
-    "import autograd.numpy as np\n",
-    "import matplotlib.pyplot as plt\n",
-    "from autograd import grad\n",
-    "\n",
-    "# Note change from previous example\n",
-    "def CostOLS(y,X,theta):\n",
-    "    return np.sum((y-X @ theta)**2)\n",
-    "\n",
-    "n = 1000\n",
-    "x = np.random.rand(n,1)\n",
-    "y = 2.0+3*x +4*x*x# +np.random.randn(n,1)\n",
-    "\n",
-    "X = np.c_[np.ones((n,1)), x, x*x]\n",
-    "XT_X = X.T @ X\n",
-    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
-    "print(\"Own inversion\")\n",
-    "print(theta_linreg)\n",
-    "\n",
-    "\n",
-    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
-    "training_gradient = grad(CostOLS,2)\n",
-    "# Define parameters for Stochastic Gradient Descent\n",
-    "n_epochs = 50\n",
-    "M = 5   #size of each minibatch\n",
-    "m = int(n/M) #number of minibatches\n",
-    "# Guess for unknown parameters theta\n",
-    "theta = np.random.randn(3,1)\n",
-    "\n",
-    "# Value for learning rate\n",
-    "eta = 0.01\n",
-    "# Value for parameter rho\n",
-    "rho = 0.99\n",
-    "# Including AdaGrad parameter to avoid possible division by zero\n",
-    "delta  = 1e-8\n",
-    "for epoch in range(n_epochs):\n",
-    "    Giter = 0.0\n",
-    "    for i in range(m):\n",
-    "        random_index = M*np.random.randint(m)\n",
-    "        xi = X[random_index:random_index+M]\n",
-    "        yi = y[random_index:random_index+M]\n",
-    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
-    "\t# Accumulated gradient\n",
-    "\t# Scaling with rho the new and the previous results\n",
-    "        Giter = (rho*Giter+(1-rho)*gradients*gradients)\n",
-    "\t# Taking the diagonal only and inverting\n",
-    "        update = gradients*eta/(delta+np.sqrt(Giter))\n",
-    "\t# Hadamard product\n",
-    "        theta -= update\n",
-    "print(\"theta from own RMSprop\")\n",
-    "print(theta)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "2e0860f7",
-   "metadata": {},
-   "source": [
-    "## ADAM algorithm, taken from [Goodfellow et al](https://www.deeplearningbook.org/contents/optimization.html)\n",
-    "\n",
-    "<!-- dom:FIGURE: [figures/adam.png, width=600 frac=0.8] -->\n",
-    "<!-- begin figure -->\n",
-    "\n",
-    "<img src=\"figures/adam.png\" width=\"600\"><p style=\"font-size: 0.9em\"><i>Figure 1: </i></p>\n",
-    "<!-- end figure -->"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "ab4a9859",
-   "metadata": {},
-   "source": [
-    "## And finally [ADAM](https://arxiv.org/pdf/1412.6980.pdf)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 9,
-   "id": "ccdd4d77",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# Using Autograd to calculate gradients using RMSprop  and Stochastic Gradient descent\n",
-    "# OLS example\n",
-    "from random import random, seed\n",
-    "import numpy as np\n",
-    "import autograd.numpy as np\n",
-    "import matplotlib.pyplot as plt\n",
-    "from autograd import grad\n",
-    "\n",
-    "# Note change from previous example\n",
-    "def CostOLS(y,X,theta):\n",
-    "    return np.sum((y-X @ theta)**2)\n",
-    "\n",
-    "n = 1000\n",
-    "x = np.random.rand(n,1)\n",
-    "y = 2.0+3*x +4*x*x# +np.random.randn(n,1)\n",
-    "\n",
-    "X = np.c_[np.ones((n,1)), x, x*x]\n",
-    "XT_X = X.T @ X\n",
-    "theta_linreg = np.linalg.pinv(XT_X) @ (X.T @ y)\n",
-    "print(\"Own inversion\")\n",
-    "print(theta_linreg)\n",
-    "\n",
-    "\n",
-    "# Note that we request the derivative wrt third argument (theta, 2 here)\n",
-    "training_gradient = grad(CostOLS,2)\n",
-    "# Define parameters for Stochastic Gradient Descent\n",
-    "n_epochs = 50\n",
-    "M = 5   #size of each minibatch\n",
-    "m = int(n/M) #number of minibatches\n",
-    "# Guess for unknown parameters theta\n",
-    "theta = np.random.randn(3,1)\n",
-    "\n",
-    "# Value for learning rate\n",
-    "eta = 0.01\n",
-    "# Value for parameters beta1 and beta2, see https://arxiv.org/abs/1412.6980\n",
-    "beta1 = 0.9\n",
-    "beta2 = 0.999\n",
-    "# Including AdaGrad parameter to avoid possible division by zero\n",
-    "delta  = 1e-7\n",
-    "iter = 0\n",
-    "for epoch in range(n_epochs):\n",
-    "    first_moment = 0.0\n",
-    "    second_moment = 0.0\n",
-    "    iter += 1\n",
-    "    for i in range(m):\n",
-    "        random_index = M*np.random.randint(m)\n",
-    "        xi = X[random_index:random_index+M]\n",
-    "        yi = y[random_index:random_index+M]\n",
-    "        gradients = (1.0/M)*training_gradient(yi, xi, theta)\n",
-    "        # Computing moments first\n",
-    "        first_moment = beta1*first_moment + (1-beta1)*gradients\n",
-    "        second_moment = beta2*second_moment+(1-beta2)*gradients*gradients\n",
-    "        first_term = first_moment/(1.0-beta1**iter)\n",
-    "        second_term = second_moment/(1.0-beta2**iter)\n",
-    "\t# Scaling with rho the new and the previous results\n",
-    "        update = eta*first_term/(np.sqrt(second_term)+delta)\n",
-    "        theta -= update\n",
-    "print(\"theta from own ADAM\")\n",
-    "print(theta)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "25ac988c",
-   "metadata": {},
-   "source": [
-    "## Introducing [JAX](https://jax.readthedocs.io/en/latest/)\n",
-    "\n",
-    "Presently, instead of using **autograd**, we recommend using [JAX](https://jax.readthedocs.io/en/latest/)\n",
-    "\n",
-    "**JAX** is Autograd and [XLA (Accelerated Linear Algebra))](https://www.tensorflow.org/xla),\n",
-    "brought together for high-performance numerical computing and machine learning research.\n",
-    "It provides composable transformations of Python+NumPy programs: differentiate, vectorize, parallelize, Just-In-Time compile to GPU/TPU, and more."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "37d556d0",
-   "metadata": {},
-   "source": [
-    "### Getting started with Jax, note the way we import numpy"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 10,
-   "id": "5b81d6e4",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import jax\n",
-    "import jax.numpy as jnp\n",
-    "import numpy as np\n",
-    "import matplotlib.pyplot as plt\n",
-    "\n",
-    "from jax import grad as jax_grad"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "c42db672",
-   "metadata": {},
-   "source": [
-    "### A warm-up example"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 11,
-   "id": "98eb2f26",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "def function(x):\n",
-    "    return x**2\n",
-    "\n",
-    "def analytical_gradient(x):\n",
-    "    return 2*x\n",
-    "\n",
-    "def gradient_descent(starting_point, learning_rate, num_iterations, solver=\"analytical\"):\n",
-    "    x = starting_point\n",
-    "    trajectory_x = [x]\n",
-    "    trajectory_y = [function(x)]\n",
-    "\n",
-    "    if solver == \"analytical\":\n",
-    "        grad = analytical_gradient    \n",
-    "    elif solver == \"jax\":\n",
-    "        grad = jax_grad(function)\n",
-    "        x = jnp.float64(x)\n",
-    "        learning_rate = jnp.float64(learning_rate)\n",
-    "\n",
-    "    for _ in range(num_iterations):\n",
-    "        \n",
-    "        x = x - learning_rate * grad(x)\n",
-    "        trajectory_x.append(x)\n",
-    "        trajectory_y.append(function(x))\n",
-    "\n",
-    "    return trajectory_x, trajectory_y\n",
-    "\n",
-    "x = np.linspace(-5, 5, 100)\n",
-    "plt.plot(x, function(x), label=\"f(x)\")\n",
-    "\n",
-    "descent_x, descent_y = gradient_descent(5, 0.1, 10, solver=\"analytical\")\n",
-    "jax_descend_x, jax_descend_y = gradient_descent(5, 0.1, 10, solver=\"jax\")\n",
-    "\n",
-    "plt.plot(descent_x, descent_y, label=\"Gradient descent\", marker=\"o\")\n",
-    "plt.plot(jax_descend_x, jax_descend_y, label=\"JAX\", marker=\"x\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "8a5f19b5",
-   "metadata": {},
-   "source": [
-    "### A more advanced example"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 12,
-   "id": "d8f5eb38",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "backend = np\n",
-    "\n",
-    "def function(x):\n",
-    "    return x*backend.sin(x**2 + 1)\n",
-    "\n",
-    "def analytical_gradient(x):\n",
-    "    return backend.sin(x**2 + 1) + 2*x**2*backend.cos(x**2 + 1)\n",
-    "\n",
-    "\n",
-    "x = np.linspace(-5, 5, 100)\n",
-    "plt.plot(x, function(x), label=\"f(x)\")\n",
-    "\n",
-    "descent_x, descent_y = gradient_descent(1, 0.01, 300, solver=\"analytical\")\n",
-    "\n",
-    "# Change the backend to JAX\n",
-    "backend = jnp\n",
-    "jax_descend_x, jax_descend_y = gradient_descent(1, 0.01, 300, solver=\"jax\")\n",
-    "\n",
-    "plt.scatter(descent_x, descent_y, label=\"Gradient descent\", marker=\"v\", s=10, color=\"red\") \n",
-    "plt.scatter(jax_descend_x, jax_descend_y, label=\"JAX\", marker=\"x\", s=5, color=\"black\")"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.9.18"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}
diff --git a/doc/LectureNotes/exercisesweek42.ipynb b/doc/LectureNotes/exercisesweek42.ipynb
deleted file mode 100644
index c24e52ac4..000000000
--- a/doc/LectureNotes/exercisesweek42.ipynb
+++ /dev/null
@@ -1,804 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "4b4c06bc",
-   "metadata": {},
-   "source": [
-    "<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
-    "doconce format html exercisesweek41.do.txt  -->\n",
-    "<!-- dom:TITLE: Exercises week 41 -->\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "bcb25e64",
-   "metadata": {},
-   "source": [
-    "# Exercises week 42\n",
-    "\n",
-    "**October 14-18, 2024**\n",
-    "\n",
-    "Date: **Deadline is Friday October 18 at midnight**\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "bb01f126",
-   "metadata": {},
-   "source": [
-    "# Overarching aims of the exercises this week\n",
-    "\n",
-    "This week, you will implement the entire feed-forward pass of a neural network! Next week you will compute the gradient of the network by implementing back-propagation manually, and by using autograd which does back-propagation for you (much easier!). Next week, you will also use the gradient to optimize the network with a gradient method! However, there is an optional exercise this week to get started on training the network and getting good results!\n",
-    "\n",
-    "We recommend that you do the exercises this week by editing and running this notebook file, as it includes some checks along the way that you have implemented the pieces of the feed-forward pass correctly, and running small parts of the code at a time will be important for understanding the methods.\n",
-    "\n",
-    "If you have trouble running a notebook, you can run this notebook in google colab instead (https://colab.research.google.com/drive/1zKibVQf-iAYaAn2-GlKfgRjHtLnPlBX4#offline=true&sandboxMode=true), an updated link will be provided on the course discord (you can also send an email to k.h.fredly@fys.uio.no if you encounter any trouble), though we recommend that you set up VSCode and your python environment to run code like this locally.\n",
-    "\n",
-    "First, here are some functions you are going to need, don't change this cell. If you are unable to import autograd, just swap in normal numpy until you want to do the final optional exercise.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "c6f61b09",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import autograd.numpy as np  # We need to use this numpy wrapper to make automatic differentiation work later\n",
-    "from sklearn import datasets\n",
-    "import matplotlib.pyplot as plt\n",
-    "from sklearn.metrics import accuracy_score\n",
-    "\n",
-    "\n",
-    "# Defining some activation functions\n",
-    "def ReLU(z):\n",
-    "    return np.where(z > 0, z, 0)\n",
-    "\n",
-    "\n",
-    "def sigmoid(z):\n",
-    "    return 1 / (1 + np.exp(-z))\n",
-    "\n",
-    "\n",
-    "def softmax(z):\n",
-    "    \"\"\"Compute softmax values for each set of scores in the rows of the matrix z.\n",
-    "    Used with batched input data.\"\"\"\n",
-    "    e_z = np.exp(z - np.max(z, axis=0))\n",
-    "    return e_z / np.sum(e_z, axis=1)[:, np.newaxis]\n",
-    "\n",
-    "\n",
-    "def softmax_vec(z):\n",
-    "    \"\"\"Compute softmax values for each set of scores in the vector z.\n",
-    "    Use this function when you use the activation function on one vector at a time\"\"\"\n",
-    "    e_z = np.exp(z - np.max(z))\n",
-    "    return e_z / np.sum(e_z)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "6248ec53",
-   "metadata": {},
-   "source": [
-    "# Exercise 1\n",
-    "\n",
-    "In this exercise you will compute the activation of the first layer. You only need to change the code in the cells right below an exercise, the rest works out of the box. Feel free to make changes and see how stuff works though!\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "37f30740",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "np.random.seed(2024)\n",
-    "\n",
-    "x = np.random.randn(2)  # network input. This is a single input with two features\n",
-    "W1 = np.random.randn(4, 2)  # first layer weights"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "4ed2cf3d",
-   "metadata": {},
-   "source": [
-    "**a)** Given the shape of the first layer weight matrix, what is the input shape of the neural network? What is the output shape of the first layer?\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "edf7217b",
-   "metadata": {},
-   "source": [
-    "**b)** Define the bias of the first layer, `b1`with the correct shape. (Run the next cell right after the previous to get the random generated values to line up with the test solution below)\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "2129c19f",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "b1 = ..."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "09e8d453",
-   "metadata": {},
-   "source": [
-    "**c)** Compute the intermediary `z1` for the first layer\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "6837119b",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "z1 = ..."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "6f71374e",
-   "metadata": {},
-   "source": [
-    "**d)** Compute the activation `a1` for the first layer using the ReLU activation function defined earlier.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "8d41ed19",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "a1 = ..."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "088710c0",
-   "metadata": {},
-   "source": [
-    "Confirm that you got the correct activation with the test below. Make sure that you define `b1` with the randn function right after you define `W1`.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "4d2f54b4",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "sol1 = np.array([0.60610368, 4.0076268, 0.0, 0.56469864])\n",
-    "\n",
-    "print(np.allclose(a1, sol1))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "7fb0cf46",
-   "metadata": {},
-   "source": [
-    "# Exercise 2\n",
-    "\n",
-    "Now we will add a layer to the network with an output of length 8 and ReLU activation.\n",
-    "\n",
-    "**a)** What is the input of the second layer? What is its shape?\n",
-    "\n",
-    "**b)** Define the weight and bias of the second layer with the right shapes.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "00063acf",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "W2 = ...\n",
-    "b2 = ..."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "5bd7d84b",
-   "metadata": {},
-   "source": [
-    "**c)** Compute the intermediary `z2` and activation `a2` for the second layer.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "2fd0383d",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "z2 = ...\n",
-    "a2 = ..."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "1b5daae5",
-   "metadata": {},
-   "source": [
-    "Confirm that you got the correct activation shape with the test below.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "f7f2f8a1",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "print(\n",
-    "    np.allclose(np.exp(len(a2)), 2980.9579870417283)\n",
-    ")  # This should evaluate to True if a2 has the correct shape :)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "3759620d",
-   "metadata": {},
-   "source": [
-    "# Exercise 3\n",
-    "\n",
-    "We often want our neural networks to have many layers of varying sizes. To avoid writing very long and error-prone code where we explicitly define and evaluate each layer we should keep all our layers in a single variable which is easy to create and use.\n",
-    "\n",
-    "**a)** Complete the function below so that it returns a list `layers` of weight and bias tuples `(W, b)` for each layer, in order, with the correct shapes that we can use later as our network parameters.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "c58f10f9",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "def create_layers(network_input_size, layer_output_sizes):\n",
-    "    layers = []\n",
-    "\n",
-    "    i_size = network_input_size\n",
-    "    for layer_output_size in layer_output_sizes:\n",
-    "        W = ...\n",
-    "        b = ...\n",
-    "        layers.append((W, b))\n",
-    "\n",
-    "        i_size = layer_output_size\n",
-    "    return layers"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "bdc0cda2",
-   "metadata": {},
-   "source": [
-    "**b)** Comple the function below so that it evaluates the intermediary `z` and activation `a` for each layer, with ReLU actication, and returns the final activation `a`. This is the complete feed-forward pass, a full neural network!\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "5262df05",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "def feed_forward_all_relu(layers, input):\n",
-    "    a = input\n",
-    "    for W, b in layers:\n",
-    "        z = ...\n",
-    "        a = ...\n",
-    "    return a"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "245adbcb",
-   "metadata": {},
-   "source": [
-    "**c)** Create a network with input size 8 and layers with output sizes 10, 16, 6, 2. Evaluate it and make sure that you get the correct size vectors along the way.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "89a8f70d",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "input_size = ...\n",
-    "layer_output_sizes = [...]\n",
-    "\n",
-    "x = np.random.rand(input_size)\n",
-    "layers = ...\n",
-    "predict = ...\n",
-    "print(predict)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "0da7fd52",
-   "metadata": {},
-   "source": [
-    "**d)** Why is a neural network with no activation functions always mathematically equivelent to a neural network with only one layer?\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "306d8b7c",
-   "metadata": {},
-   "source": [
-    "# Exercise 4 - Custom activation for each layer\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "221c7b6c",
-   "metadata": {},
-   "source": [
-    "So far, every layer has used the same activation, ReLU. We often want to use other types of activation however, so we need to update our code to support multiple types of activation functions. Make sure that you have completed every previous exercise before trying this one.\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "10896d06",
-   "metadata": {},
-   "source": [
-    "**a)** Complete the `feed_forward` function which accepts a list of activation functions as an argument, and which evaluates these activation functions at each layer.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "de062369",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "def feed_forward(input, layers, activation_funcs):\n",
-    "    a = input\n",
-    "    for (W, b), activation_func in zip(layers, activation_funcs):\n",
-    "        z = ...\n",
-    "        a = ...\n",
-    "    return a"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "8f7df363",
-   "metadata": {},
-   "source": [
-    "**b)** You are now given a list with three activation functions, two ReLU and one sigmoid. (Don't call them yet! you can make a list with function names as elements, and then call these elements of the list later. If you add other functions than the ones defined at the start of the notebook, make sure everything is defined using autograd's numpy wrapper, like above, since we want to use automatic differentiation on all of these functions later.)\n",
-    "\n",
-    "Evaluate a network with three layers and these activation functions.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "301b46dc",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "network_input_size = ...\n",
-    "layer_output_sizes = [...]\n",
-    "activation_funcs = [ReLU, ReLU, sigmoid]\n",
-    "layers = ...\n",
-    "\n",
-    "x = np.random.randn(network_input_size)\n",
-    "feed_forward(x, layers, activation_funcs)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "9c914fd0",
-   "metadata": {},
-   "source": [
-    "**c)** How does the output of the network change if you use sigmoid in the hidden layers and ReLU in the output layer?\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "a8d6c425",
-   "metadata": {},
-   "source": [
-    "# Exercise 5 - Processing multiple inputs at once\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "0f4330a4",
-   "metadata": {},
-   "source": [
-    "So far, the feed forward function has taken one input vector as an input. This vector then undergoes a linear transformation and then an element-wise non-linear operation for each layer. This approach of sending one vector in at a time is great for interpreting how the network transforms data with its linear and non-linear operations, but not the best for numerical efficiency. Now, we want to be able to send many inputs through the network at once. This will make the code a bit harder to understand, but it will make it faster, and more compact. It will be worth the trouble.\n",
-    "\n",
-    "To process multiple inputs at once, while still performing the same operations, you will only need to flip a couple things around.\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "17023bb7",
-   "metadata": {},
-   "source": [
-    "**a)** Complete the function `create_layers_batch` so that the weight matrix is the transpose of what it was when you only sent in one input at a time.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "a241fd79",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "def create_layers_batch(network_input_size, layer_output_sizes):\n",
-    "    layers = []\n",
-    "\n",
-    "    i_size = network_input_size\n",
-    "    for layer_output_size in layer_output_sizes:\n",
-    "        W = ...\n",
-    "        b = ...\n",
-    "        layers.append((W, b))\n",
-    "\n",
-    "        i_size = layer_output_size\n",
-    "    return layers"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "a6349db6",
-   "metadata": {},
-   "source": [
-    "**b)** Make a matrix of inputs with the shape (number of features, number of inputs), you choose the number of inputs and features per input. Then complete the function `feed_forward_batch` so that you can process this matrix of inputs with only one matrix multiplication and one broadcasted vector addition per layer. (Hint: You will only need to swap two variable around from your previous implementation, but remember to test that you get the same results for equivelent inputs!)\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "425f3bcc",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "inputs = np.random.rand(1000, 4)\n",
-    "\n",
-    "\n",
-    "def feed_forward_batch(inputs, layers, activation_funcs):\n",
-    "    a = inputs\n",
-    "    for (W, b), activation_func in zip(layers, activation_funcs):\n",
-    "        z = ...\n",
-    "        a = ...\n",
-    "    return a"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "efd07b4e",
-   "metadata": {},
-   "source": [
-    "**c)** Create and evaluate a neural network with 4 inputs and layers with output sizes 12, 10, 3 and activations ReLU, ReLU, softmax.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "ce6fcc2f",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "network_input_size = ...\n",
-    "layer_output_sizes = [...]\n",
-    "activation_funcs = [...]\n",
-    "layers = create_layers_batch(network_input_size, layer_output_sizes)\n",
-    "\n",
-    "x = np.random.randn(network_input_size)\n",
-    "feed_forward_batch(inputs, layers, activation_funcs)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "87999271",
-   "metadata": {},
-   "source": [
-    "You should use this batched approach moving forward, as it will lead to much more compact code. However, remember that each input is still treated separately, and that you will need to keep in mind the transposed weight matrix and other details when implementing backpropagation.\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "237eb782",
-   "metadata": {},
-   "source": [
-    "# Exercise 6 - Predicting on real data\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "54d5fde7",
-   "metadata": {},
-   "source": [
-    "You will now evaluate your neural network on the iris data set (https://scikit-learn.org/1.5/auto_examples/datasets/plot_iris_dataset.html).\n",
-    "\n",
-    "This dataset contains data on 150 flowers of 3 different types which can be separated pretty well using the four features given for each flower, which includes the width and length of their leaves. You are will later train your network to actually make good predictions.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "6bd4c148",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "iris = datasets.load_iris()\n",
-    "\n",
-    "_, ax = plt.subplots()\n",
-    "scatter = ax.scatter(iris.data[:, 0], iris.data[:, 1], c=iris.target)\n",
-    "ax.set(xlabel=iris.feature_names[0], ylabel=iris.feature_names[1])\n",
-    "_ = ax.legend(\n",
-    "    scatter.legend_elements()[0], iris.target_names, loc=\"lower right\", title=\"Classes\"\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "ed3e2fc9",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "inputs = iris.data\n",
-    "\n",
-    "# Since each prediction is a vector with a score for each of the three types of flowers,\n",
-    "# we need to make each target a vector with a 1 for the correct flower and a 0 for the others.\n",
-    "targets = np.zeros((len(iris.data), 3))\n",
-    "for i, t in enumerate(iris.target):\n",
-    "    targets[i, t] = 1\n",
-    "\n",
-    "\n",
-    "def accuracy(predictions, targets):\n",
-    "    one_hot_predictions = np.zeros(predictions.shape)\n",
-    "\n",
-    "    for i, prediction in enumerate(predictions):\n",
-    "        one_hot_predictions[i, np.argmax(prediction)] = 1\n",
-    "    return accuracy_score(one_hot_predictions, targets)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "0362c4a9",
-   "metadata": {},
-   "source": [
-    "**a)** What should the input size for the network be with this dataset? What should the output size of the last layer be?\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "bf62607e",
-   "metadata": {},
-   "source": [
-    "**b)** Create a network with two hidden layers, the first with sigmoid activation and the last with softmax, the first layer should have 8 \"nodes\", the second has the number of nodes you found in exercise a). Softmax returns a \"probability distribution\", in the sense that the numbers in the output are positive and add up to 1 and, their magnitude are in some sense relative to their magnitude before going through the softmax function. Remember to use the batched version of the create_layers and feed forward functions.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "5366d4ae",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "...\n",
-    "layers = ..."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "c528846f",
-   "metadata": {},
-   "source": [
-    "**c)** Evaluate your model on the entire iris dataset! For later purposes, we will split the data into train and test sets, and compute gradients on smaller batches of the training data. But for now, evaluate the network on the whole thing at once.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "6c783105",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "predictions = feed_forward_batch(inputs, layers, activation_funcs)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "01a3caa8",
-   "metadata": {},
-   "source": [
-    "**d)** Compute the accuracy of your model using the accuracy function defined above. Recreate your model a couple times and see how the accuracy changes.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "a2612b82",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "print(accuracy(predictions, targets))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "334560b6",
-   "metadata": {},
-   "source": [
-    "# Exercise 7 - Training on real data (Optional)\n",
-    "\n",
-    "To be able to actually do anything useful with your neural network, you need to train it. For this, we need a cost function and a way to take the gradient of the cost function wrt. the network parameters. The following exercises guide you through taking the gradient using autograd, and updating the network parameters using the gradient. Feel free to implement gradient methods like ADAM if you finish everything.\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "700cabe4",
-   "metadata": {},
-   "source": [
-    "Since we are doing a classification task with multiple output classes, we use the cross-entropy loss function, which can evaluate performance on classification tasks. It sees if your prediction is \"most certain\" on the correct target.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "f30e6e2c",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "def cross_entropy(predict, target):\n",
-    "    return np.sum(-target * np.log(predict))\n",
-    "\n",
-    "\n",
-    "def cost(input, layers, activation_funcs, target):\n",
-    "    predict = feed_forward_batch(input, layers, activation_funcs)\n",
-    "    return cross_entropy(predict, target)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "7ea9c1a4",
-   "metadata": {},
-   "source": [
-    "To improve our network on whatever prediction task we have given it, we need to use a sensible cost function, take the gradient of that cost function with respect to our network parameters, the weights and biases, and then update the weights and biases using these gradients. To clarify, we need to find and use these\n",
-    "\n",
-    "$$\n",
-    "\\frac{\\partial C}{\\partial W}, \\frac{\\partial C}{\\partial b}\n",
-    "$$\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "6c753e3b",
-   "metadata": {},
-   "source": [
-    "Now we need to compute these gradients. This is pretty hard to do for a neural network, we will use most of next week to do this, but we can also use autograd to just do it for us, which is what we always do in practice. With the code cell below, we create a function which takes all of these gradients for us.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "56bef776",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from autograd import grad\n",
-    "\n",
-    "\n",
-    "gradient_func = grad(\n",
-    "    cost, 1\n",
-    ")  # Taking the gradient wrt. the second input to the cost function, i.e. the layers"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "7b1b74bc",
-   "metadata": {},
-   "source": [
-    "**a)** What shape should the gradient of the cost function wrt. weights and biases be?\n",
-    "\n",
-    "**b)** Use the `gradient_func` function to take the gradient of the cross entropy wrt. the weights and biases of the network. Check the shapes of what's inside. What does the `grad` func from autograd actually do?\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "841c9e87",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "layers_grad = gradient_func(\n",
-    "    inputs, layers, activation_funcs, targets\n",
-    ")  # Don't change this"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "adc9e9be",
-   "metadata": {},
-   "source": [
-    "**c)** Finish the `train_network` function.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "6e4d38d3",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "def train_network(\n",
-    "    inputs, layers, activation_funcs, targets, learning_rate=0.001, epochs=100\n",
-    "):\n",
-    "    for i in range(epochs):\n",
-    "        layers_grad = gradient_func(inputs, layers, activation_funcs, targets)\n",
-    "        for (W, b), (W_g, b_g) in zip(layers, layers_grad):\n",
-    "            W -= ...\n",
-    "            b -= ..."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "2f65d663",
-   "metadata": {},
-   "source": [
-    "**e)** What do we call the gradient method used above?\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "7059dd8c",
-   "metadata": {},
-   "source": [
-    "**d)** Train your network and see how the accuracy changes! Make a plot if you want.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "5027c7a5",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "..."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "3bc77016",
-   "metadata": {},
-   "source": [
-    "**e)** How high of an accuracy is it possible to acheive with a neural network on this dataset, if we use the whole thing as training data?\n"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.9.15"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}
diff --git a/doc/LectureNotes/exercisesweek43.ipynb b/doc/LectureNotes/exercisesweek43.ipynb
deleted file mode 100644
index c0b6e5f7b..000000000
--- a/doc/LectureNotes/exercisesweek43.ipynb
+++ /dev/null
@@ -1,717 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Exercises week 43\n",
-    "\n",
-    "**October 18-25, 2024**\n",
-    "\n",
-    "Date: **Deadline is Friday October 25 at midnight**\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Overarching aims of the exercises this week\n",
-    "\n",
-    "The aim of the exercises this week is to train the neural network you implemented last week.\n",
-    "\n",
-    "To train neural networks, we use gradient descent, since there is no analytical expression for the optimal parameters. This means you will need to compute the gradient of the cost function wrt. the network parameters. And then you will need to implement some gradient method.\n",
-    "\n",
-    "You will begin by computing gradients for a network with one layer, then two layers, then any number of layers. Keeping track of the shapes and doing things step by step will be very important this week.\n",
-    "\n",
-    "We recommend that you do the exercises this week by editing and running this notebook file, as it includes some checks along the way that you have implemented the neural network correctly, and running small parts of the code at a time will be important for understanding the methods. If you have trouble running a notebook, you can run this notebook in google colab instead(https://colab.research.google.com/drive/1FfvbN0XlhV-lATRPyGRTtTBnJr3zNuHL#offline=true&sandboxMode=true), though we recommend that you set up VSCode and your python environment to run code like this locally.\n",
-    "\n",
-    "First, some setup code that you will need.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 3,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "import autograd.numpy as np  # We need to use this numpy wrapper to make automatic differentiation work later\n",
-    "from autograd import grad, elementwise_grad\n",
-    "from sklearn import datasets\n",
-    "import matplotlib.pyplot as plt\n",
-    "from sklearn.metrics import accuracy_score\n",
-    "\n",
-    "\n",
-    "# Defining some activation functions\n",
-    "def ReLU(z):\n",
-    "    return np.where(z > 0, z, 0)\n",
-    "\n",
-    "\n",
-    "# Derivative of the ReLU function\n",
-    "def ReLU_der(z):\n",
-    "    return np.where(z > 0, 1, 0)\n",
-    "\n",
-    "\n",
-    "def sigmoid(z):\n",
-    "    return 1 / (1 + np.exp(-z))\n",
-    "\n",
-    "\n",
-    "def mse(predict, target):\n",
-    "    return np.mean((predict - target) ** 2)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Exercise 1 - Understand the feed forward pass\n",
-    "\n",
-    "**a)** Complete last weeks' mandatory exercises if you haven't already.\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Exercise 2 - Gradient with one layer using autograd\n",
-    "\n",
-    "For the first few exercises, we will not use batched inputs. Only a single input vector is passed through the layer at a time.\n",
-    "\n",
-    "In this exercise you will compute the gradient of a single layer. You only need to change the code in the cells right below an exercise, the rest works out of the box. Feel free to make changes and see how stuff works though!\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "**a)** If the weights and bias of a layer has shapes (10, 4) and (10), what will the shapes of the gradients of the cost function wrt. these weights and this bias be?\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "**b)** Complete the feed_forward_one_layer function. It should use the sigmoid activation function. Also define the weigth and bias with the correct shapes.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 41,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "def feed_forward_one_layer(W, b, x):\n",
-    "    z = ...\n",
-    "    a = ...\n",
-    "    return a\n",
-    "\n",
-    "\n",
-    "def cost_one_layer(W, b, x, target):\n",
-    "    predict = feed_forward_one_layer(W, b, x)\n",
-    "    return mse(predict, target)\n",
-    "\n",
-    "\n",
-    "x = np.random.rand(2)\n",
-    "target = np.random.rand(3)\n",
-    "\n",
-    "W = ...\n",
-    "b = ..."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "**c)** Compute the gradient of the cost function wrt. the weigth and bias by running the cell below. You will not need to change anything, just make sure it runs by defining things correctly in the cell above. This code uses the autograd package which uses backprogagation to compute the gradient!\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "autograd_one_layer = grad(cost_one_layer, [0, 1])\n",
-    "W_g, b_g = autograd_one_layer(W, b, x, target)\n",
-    "print(W_g, b_g)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Exercise 3 - Gradient with one layer writing backpropagation by hand\n",
-    "\n",
-    "Before you use the gradient you found using autograd, you will have to find the gradient \"manually\", to better understand how the backpropagation computation works. To do backpropagation \"manually\", you will need to write out expressions for many derivatives along the computation.\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "We want to find the gradient of the cost function wrt. the weight and bias. This is quite hard to do directly, so we instead use the chain rule to combine multiple derivatives which are easier to compute.\n",
-    "\n",
-    "$$\n",
-    "\\frac{dC}{dW} = \\frac{dC}{da}\\frac{da}{dz}\\frac{dz}{dW}\n",
-    "$$\n",
-    "\n",
-    "$$\n",
-    "\\frac{dC}{db} = \\frac{dC}{da}\\frac{da}{dz}\\frac{dz}{db}\n",
-    "$$\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "**a)** Which intermediary results can be reused between the two expressions?\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "**b)** What is the derivative of the cost wrt. the final activation? You can use the autograd calculation to make sure you get the correct result. Remember that we compute the mean in mse.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "z = W @ x + b\n",
-    "a = sigmoid(z)\n",
-    "\n",
-    "predict = a\n",
-    "\n",
-    "\n",
-    "def mse_der(predict, target):\n",
-    "    return ...\n",
-    "\n",
-    "\n",
-    "print(mse_der(predict, target))\n",
-    "\n",
-    "cost_autograd = grad(mse, 0)\n",
-    "print(cost_autograd(predict, target))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "**c)** What is the expression for the derivative of the sigmoid activation function? You can use the autograd calculation to make sure you get the correct result.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "def sigmoid_der(z):\n",
-    "    return ...\n",
-    "\n",
-    "\n",
-    "print(sigmoid_der(z))\n",
-    "\n",
-    "sigmoid_autograd = elementwise_grad(sigmoid, 0)\n",
-    "print(sigmoid_autograd(z))"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "**d)** Using the two derivatives you just computed, compute this intermetidary gradient you will use later:\n",
-    "\n",
-    "$$\n",
-    "\\frac{dC}{dz} = \\frac{dC}{da}\\frac{da}{dz}\n",
-    "$$\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 54,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "dC_da = ...\n",
-    "dC_dz = ..."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "**e)** What is the derivative of the intermediary z wrt. the weight and bias? What should the shapes be? The one for the weights is a little tricky, it can be easier to play around in the next exercise first. You can also try computing it with autograd to get a hint.\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "**f)** Now combine the expressions you have worked with so far to compute the gradients! Note that you always need to do a feed forward pass while saving the zs and as before you do backpropagation, as they are used in the derivative expressions\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "dC_da = ...\n",
-    "dC_dz = ...\n",
-    "dC_dW = ...\n",
-    "dC_db = ...\n",
-    "\n",
-    "print(dC_dW, dC_db)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "You should get the same results as with autograd.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "W_g, b_g = autograd_one_layer(W, b, x, target)\n",
-    "print(W_g, b_g)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Exercise 4 - Gradient with two layers writing backpropagation by hand\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Now that you have implemented backpropagation for one layer, you have found most of the expressions you will need for more layers. Let's move up to two layers.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 59,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "x = np.random.rand(2)\n",
-    "target = np.random.rand(4)\n",
-    "\n",
-    "W1 = np.random.rand(3, 2)\n",
-    "b1 = np.random.rand(3)\n",
-    "\n",
-    "W2 = np.random.rand(4, 3)\n",
-    "b2 = np.random.rand(4)\n",
-    "\n",
-    "layers = [(W1, b1), (W2, b2)]"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 60,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "z1 = W1 @ x + b1\n",
-    "a1 = sigmoid(z1)\n",
-    "z2 = W2 @ a1 + b2\n",
-    "a2 = sigmoid(z2)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "We begin by computing the gradients of the last layer, as the gradients must be propagated backwards from the end.\n",
-    "\n",
-    "**a)** Compute the gradients of the last layer, just like you did the single layer in the previous exercise.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 61,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "dC_da2 = ...\n",
-    "dC_dz2 = ...\n",
-    "dC_dW2 = ...\n",
-    "dC_db2 = ..."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "To find the derivative of the cost wrt. the activation of the first layer, we need a new expression, the one furthest to the right in the following.\n",
-    "\n",
-    "$$\n",
-    "\\frac{dC}{da_1} = \\frac{dC}{dz_2}\\frac{dz_2}{da_1}\n",
-    "$$\n",
-    "\n",
-    "**b)** What is the derivative of the second layer intermetiate wrt. the first layer activation? (First recall how you compute $z_2$)\n",
-    "\n",
-    "$$\n",
-    "\\frac{dz_2}{da_1}\n",
-    "$$\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": []
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "**c)** Use this expression, together with expressions which are equivelent to ones for the last layer to compute all the derivatives of the first layer.\n",
-    "\n",
-    "$$\n",
-    "\\frac{dC}{dW_1} = \\frac{dC}{da_1}\\frac{da_1}{dz_1}\\frac{dz_1}{dW_1}\n",
-    "$$\n",
-    "\n",
-    "$$\n",
-    "\\frac{dC}{db_1} = \\frac{dC}{da_1}\\frac{da_1}{dz_1}\\frac{dz_1}{db_1}\n",
-    "$$\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 63,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "dC_da1 = ...\n",
-    "dC_dz1 = ...\n",
-    "dC_dW1 = ...\n",
-    "dC_db1 = ..."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "print(dC_dW1, dC_db1)\n",
-    "print(dC_dW2, dC_db2)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "**d)** Make sure you got the same gradient as the following code which uses autograd to do backpropagation.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 67,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "def feed_forward_two_layers(layers, x):\n",
-    "    W1, b1 = layers[0]\n",
-    "    z1 = W1 @ x + b1\n",
-    "    a1 = sigmoid(z1)\n",
-    "\n",
-    "    W2, b2 = layers[1]\n",
-    "    z2 = W2 @ a1 + b2\n",
-    "    a2 = sigmoid(z2)\n",
-    "\n",
-    "    return a2"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "def cost_two_layers(layers, x, target):\n",
-    "    predict = feed_forward_two_layers(layers, x)\n",
-    "    return mse(predict, target)\n",
-    "\n",
-    "\n",
-    "grad_two_layers = grad(cost_two_layers, 0)\n",
-    "grad_two_layers(layers, x, target)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "**e)** How would you use the gradient from this layer to compute the gradient of an even earlier layer? Would the expressions be any different?\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Exercise 5 - Gradient with any number of layers writing backpropagation by hand\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "Well done on getting this far! Now it's time to compute the gradient with any number of layers.\n",
-    "\n",
-    "First, some code from the general neural network code from last week. Note that we are still sending in one input vector at a time. We will change it to use batched inputs later.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 4,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "def create_layers(network_input_size, layer_output_sizes):\n",
-    "    layers = []\n",
-    "\n",
-    "    i_size = network_input_size\n",
-    "    for layer_output_size in layer_output_sizes:\n",
-    "        W = np.random.randn(layer_output_size, i_size)\n",
-    "        b = np.random.randn(layer_output_size)\n",
-    "        layers.append((W, b))\n",
-    "\n",
-    "        i_size = layer_output_size\n",
-    "    return layers\n",
-    "\n",
-    "\n",
-    "def feed_forward(input, layers, activation_funcs):\n",
-    "    a = input\n",
-    "    for (W, b), activation_func in zip(layers, activation_funcs):\n",
-    "        z = W @ a + b\n",
-    "        a = activation_func(z)\n",
-    "    return a\n",
-    "\n",
-    "\n",
-    "def cost(layers, input, activation_funcs, target):\n",
-    "    predict = feed_forward(input, layers, activation_funcs)\n",
-    "    return mse(predict, target)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "You might have already have noticed a very important detail in backpropagation: You need the values from the forward pass to compute all the gradients! The feed forward method above is great for efficiency and for using autograd, as it only cares about computing the final output, but now we need to also save the results along the way.\n",
-    "\n",
-    "Here is a function which does that for you.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 5,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "def feed_forward_saver(input, layers, activation_funcs):\n",
-    "    layer_inputs = []\n",
-    "    zs = []\n",
-    "    a = input\n",
-    "    for (W, b), activation_func in zip(layers, activation_funcs):\n",
-    "        layer_inputs.append(a)\n",
-    "        z = W @ a + b\n",
-    "        a = activation_func(z)\n",
-    "\n",
-    "        zs.append(z)\n",
-    "\n",
-    "    return layer_inputs, zs, a"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "**a)** Now, complete the backpropagation function so that it returns the gradient of the cost function wrt. all the weigths and biases. Use the autograd calculation below to make sure you get the correct answer.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "def backpropagation(\n",
-    "    input, layers, activation_funcs, target, activation_ders, cost_der=mse_der\n",
-    "):\n",
-    "    layer_inputs, zs, predict = feed_forward_saver(input, layers, activation_funcs)\n",
-    "\n",
-    "    layer_grads = [() for layer in layers]\n",
-    "\n",
-    "    # We loop over the layers, from the last to the first\n",
-    "    for i in reversed(range(len(layers))):\n",
-    "        layer_input, z, activation_der = layer_inputs[i], zs[i], activation_ders[i]\n",
-    "\n",
-    "        if i == len(layers) - 1:\n",
-    "            # For last layer we use cost derivative as dC_da(L) can be computed directly\n",
-    "            dC_da = ...\n",
-    "        else:\n",
-    "            # For other layers we build on previous z derivative, as dC_da(i) = dC_dz(i+1) * dz(i+1)_da(i)\n",
-    "            (W, b) = layers[i + 1]\n",
-    "            dC_da = ...\n",
-    "\n",
-    "        dC_dz = ...\n",
-    "        dC_dW = ...\n",
-    "        dC_db = ...\n",
-    "\n",
-    "        layer_grads[i] = (dC_dW, dC_db)\n",
-    "\n",
-    "    return layer_grads"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "network_input_size = 2\n",
-    "layer_output_sizes = [3, 4]\n",
-    "activation_funcs = [sigmoid, ReLU]\n",
-    "activation_ders = [sigmoid_der, ReLU_der]\n",
-    "\n",
-    "layers = create_layers(network_input_size, layer_output_sizes)\n",
-    "\n",
-    "x = np.random.rand(network_input_size)\n",
-    "target = np.random.rand(4)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "layer_grads = backpropagation(x, layers, activation_funcs, target, activation_ders)\n",
-    "print(layer_grads)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "cost_grad = grad(cost, 0)\n",
-    "cost_grad(layers, x, [sigmoid, ReLU], target)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Exercise 6 - Batched inputs\n",
-    "\n",
-    "Make new versions of all the functions in exercise 5 which now take batched inputs instead. See last weeks exercise 5 for details on how to batch inputs to neural networks. You will also need to update the backpropogation function.\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Exercise 7 - Training\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "**a)** Complete exercise 6 and 7 from last week, but use your own backpropogation implementation to compute the gradient.\n",
-    "\n",
-    "**b)** Use stochastic gradient descent with momentum when you train your network.\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "# Exercise 8 (Optional) - Object orientation\n",
-    "\n",
-    "Passing in the layers, activations functions, activation derivatives and cost derivatives into the functions each time leads to code which is easy to understand in isoloation, but messier when used in a larger context with data splitting, data scaling, gradient methods and so forth. Creating an object which stores these values can lead to code which is much easier to use.\n",
-    "\n",
-    "**a)** Write a neural network class. You are free to implement it how you see fit, though we strongly recommend to not save any input or output values as class attributes, nor let the neural network class handle gradient methods internally. Gradient methods should be handled outside, by performing general operations on the layer_grads list using functions or classes separate to the neural network.\n",
-    "\n",
-    "We provide here a skeleton structure which should get you started.\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "class NeuralNetwork:\n",
-    "    def __init__(\n",
-    "        self,\n",
-    "        network_input_size,\n",
-    "        layer_output_sizes,\n",
-    "        activation_funcs,\n",
-    "        activation_ders,\n",
-    "        cost_fun,\n",
-    "        cost_der,\n",
-    "    ):\n",
-    "        pass\n",
-    "\n",
-    "    def predict(self, inputs):\n",
-    "        # Simple feed forward pass\n",
-    "        pass\n",
-    "\n",
-    "    def cost(self, inputs, targets):\n",
-    "        pass\n",
-    "\n",
-    "    def _feed_forward_saver(self, inputs):\n",
-    "        pass\n",
-    "\n",
-    "    def compute_gradient(self, inputs, targets):\n",
-    "        pass\n",
-    "\n",
-    "    def update_weights(self, layer_grads):\n",
-    "        pass\n",
-    "\n",
-    "    # These last two methods are not needed in the project, but they can be nice to have! The first one has a layers parameter so that you can use autograd on it\n",
-    "    def autograd_compliant_predict(self, layers, inputs):\n",
-    "        pass\n",
-    "\n",
-    "    def autograd_gradient(self, inputs, targets):\n",
-    "        pass"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": ".venv",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.12.7"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 2
-}
diff --git a/doc/LectureNotes/exercisesweek47.ipynb b/doc/LectureNotes/exercisesweek47.ipynb
deleted file mode 100644
index 143336f25..000000000
--- a/doc/LectureNotes/exercisesweek47.ipynb
+++ /dev/null
@@ -1,153 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "73c0b63f",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
-    "doconce format html exercisesweek47.do.txt  -->\n",
-    "<!-- dom:TITLE: Exercise week 47 -->"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "8cb10d9d",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "# Exercise week 47\n",
-    "**November 18-22, 2024**\n",
-    "\n",
-    "Date: **Deadline is Friday November 22 at midnight**"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "2b90add8",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "# Overarching aims of the exercises this week\n",
-    "\n",
-    "The exercise set this week is meant as a summary of many of the\n",
-    "central elements in various machine learning algorithms, with a slight\n",
-    "bias towards deep learning methods and their training. You don't need to answer all questions.\n",
-    "\n",
-    "The last weekly exercise (week 48) is a general course survey."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "b4f3ae78",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "## Exercise 1: Linear and logistic regression methods\n",
-    "\n",
-    "1. What is the main difference between ordinary least squares and Ridge regression?\n",
-    "\n",
-    "2. Which kind of data set would you use logistic regression for?\n",
-    "\n",
-    "3. In linear regression you assume that your output is described by a continuous non-stochastic function $f(x)$. Which is the equivalent function in logistic regression?\n",
-    "\n",
-    "4. Can you find an analytic solution to a logistic regression type of problem?\n",
-    "\n",
-    "5. What kind of cost function would you use in logistic regression?"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "755cfd27",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "## Exercise 2: Deep learning\n",
-    "\n",
-    "1. What is an activation function and discuss the use of an activation function? Explain three different types of activation functions?\n",
-    "\n",
-    "2. Describe the architecture of a typical feed forward  Neural Network (NN). \n",
-    "\n",
-    "3. You are using a deep neural network for a prediction task. After training your model, you notice that it is strongly overfitting the training set and that the performance on the test isn’t good. What can you do to reduce overfitting?\n",
-    "\n",
-    "4. How would you know if your model is suffering from the problem of exploding Gradients?\n",
-    "\n",
-    "5. Can you name and explain a few hyperparameters used for training a neural network?\n",
-    "\n",
-    "6. Describe the architecture of a typical Convolutional Neural Network (CNN)\n",
-    "\n",
-    "7. What is the vanishing gradient problem in Neural Networks and how to fix it?\n",
-    "\n",
-    "8. When it comes to training an artificial neural network, what could the reason be for why the cost/loss doesn't decrease in a few epochs?\n",
-    "\n",
-    "9. How does L1/L2 regularization affect a neural network?\n",
-    "\n",
-    "10. What is(are) the advantage(s) of deep learning over traditional methods like linear regression or logistic regression?"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "85175b87",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "## Exercise 3: Decision trees and ensemble methods\n",
-    "\n",
-    "1. Mention some pros and cons when using decision trees\n",
-    "\n",
-    "2. How do we grow a tree? And which are the main parameters? \n",
-    "\n",
-    "3. Mention some of the benefits with using ensemble methods (like bagging, random forests and boosting methods)?\n",
-    "\n",
-    "4. Why would you prefer a random forest instead of using Bagging to grow a forest?\n",
-    "\n",
-    "5. What is the basic philosophy behind boosting methods?"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "fbfdfe68",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "## Exercise 4: Optimization part\n",
-    "\n",
-    "1. Which is the basic mathematical root-finding method behind essentially all gradient descent approaches(stochastic and non-stochastic)? \n",
-    "\n",
-    "2. And why don't we use it? Or stated differently, why do we introduce the learning rate as a parameter?\n",
-    "\n",
-    "3. What might happen if you set the momentum hyperparameter too close to 1 (e.g., 0.9999) when using an optimizer for the learning rate?\n",
-    "\n",
-    "4. Why should we use stochastic gradient descent instead of plain gradient descent?\n",
-    "\n",
-    "5. Which parameters would you need to tune when use a stochastic gradient descent approach?"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "92fc1b0c",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "## Exercise 5: Analysis of results\n",
-    "1. How do you assess overfitting and underfitting?\n",
-    "\n",
-    "2. Why do we divide the data in test and train and/or eventually validation sets?\n",
-    "\n",
-    "3. Why would you use resampling methods in the data analysis? Mention some widely popular resampling methods."
-   ]
-  }
- ],
- "metadata": {},
- "nbformat": 4,
- "nbformat_minor": 5
-}
diff --git a/doc/LectureNotes/exercisesweek48.ipynb b/doc/LectureNotes/exercisesweek48.ipynb
deleted file mode 100644
index 96805589d..000000000
--- a/doc/LectureNotes/exercisesweek48.ipynb
+++ /dev/null
@@ -1,229 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "30071d16",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
-    "doconce format html exercisesweek48.do.txt  -->\n",
-    "<!-- dom:TITLE: Exercises week 48 -->"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "268eaea0",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "# Exercises week 48\n",
-    "**November 25-29, 2024**\n",
-    "\n",
-    "Date: **Deadline is Friday November 29 at midnight**"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "1e04cbee",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "# Overarching aims of the exercises this week\n",
-    "\n",
-    "The exercise this week is a simple course survey and feedback.  This\n",
-    "is important for us in order to improve our teaching material, the\n",
-    "active learning format and anything else related to a succesful\n",
-    "mastering of central machine learning methods and their applications."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "78363328",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "### Why did you choose this course?"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "d4950ede",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "### What was your programming knowledge before you started?\n",
-    "\n",
-    "And do you feel this course added to your programming competences and skills?"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "95e23f0c",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "### How do you judge your own level of knowledge on machine learning before and after this course?\n",
-    "\n",
-    "Here you can discuss your level of skill/knowledge at the start of course\n",
-    "and at the end of the course and how these matched the level of\n",
-    "skill/knowledge needed to complete the projects."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "db819721",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "### Did the projects and the teaching material allow you to deepen your insights about Machine Learning methods?\n",
-    "\n",
-    "Feel free to comment here."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "61d153a4",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "### Project based teaching and active learning\n",
-    "\n",
-    "This is a project based course and we as teachers would like to keep\n",
-    "it as it is since we see very clearly that people who attend this\n",
-    "course have a very good learning outcome. Project based courses are\n",
-    "however demanding (and expensive seen from the university admin) when\n",
-    "it comes to proper feedback and evaluations. Feel free to discuss\n",
-    "whether you found a project-based useful. Feel also free to comment upon things we can improve upon or\n",
-    "alternative ways to assess whether the learning outcomes have been\n",
-    "achieved. Would you for example prefer a standard 4 hours written exam\n",
-    "be something you would prefer?  Or other alternatives to projects? We\n",
-    "would very much value your thoughts here since projects are an\n",
-    "essential part of this course."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "ee1af3ff",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "### Usefulness of the weekly exercises\n",
-    "\n",
-    "Did the weekly exercises help in getting started with the projects?\n",
-    "How relevant where they for solving the projects?  Feel free to\n",
-    "elaborate"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "46a35899",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "### Lab sessions and lectures\n",
-    "\n",
-    "Was there a good link between lectures and lab sessions?\n",
-    "Feel\n",
-    "free to comment."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "f6524f03",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "### How would you improve this course?\n",
-    "\n",
-    "Are there topics which are missing, topics which could have been\n",
-    "omitted and/or discussed in more depth? Feel free to add your comments\n",
-    "here such as how to improve to teaching material and more."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "ab89b772",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "## Then some basic questions"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "9811e1b4",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "### Which is your preferred information chanel, Canvas, Discord, mail or other?"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "6a3741fa",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "### Was the weekly update with plans etc useful?"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "a51df0b9",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "### Was it easy to access the course material?"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "762d5153",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "### Which resources and tools did you use? Jupyter-notebooks, GitHub, the various textbooks we have recommended, etc etc"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "2bf11aae",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "### If you did not attend the lectures or the lab sessions, which resources did you use?"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "3f92f5ff",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "### Any other topics, impressions, ideas etc you would like to share with us?"
-   ]
-  }
- ],
- "metadata": {},
- "nbformat": 4,
- "nbformat_minor": 5
-}
diff --git a/doc/src/week34/week34.do.txt b/doc/src/week34/week34.do.txt
index 9613fc65f..1ce863d6d 100644
--- a/doc/src/week34/week34.do.txt
+++ b/doc/src/week34/week34.do.txt
@@ -15,7 +15,7 @@ o There are four groups:
 
 o On Mondays  we have a regular lecture which will be organized as a mix of active learning sessions and regular lectures. These lectures/active learning sessions start at 215pm and end at 4pm and serve the aims of giving an overview over various topics as well as solving specific problems. These lectures will also be recorded. Lectures can be attended in person or via zoom at URL:"https://uio.zoom.us/my/mortenhj"
 #  * "Link to recording of lecture":"https://youtu.be/82IPtCrzbhs"
-The labs are also available till 6pm Tuesdays and Wednesdays. Videos and learning material with reading suggestions will be made available before each week starts.
+Videos and learning material with reading suggestions will be made available before each week starts.