diff --git a/doc/pub/week40/html/figures/figures/adagrad.png b/doc/pub/week40/html/figures/figures/adagrad.png
new file mode 100644
index 000000000..97a9cf908
Binary files /dev/null and b/doc/pub/week40/html/figures/figures/adagrad.png differ
diff --git a/doc/pub/week40/html/figures/figures/adam.png b/doc/pub/week40/html/figures/figures/adam.png
new file mode 100644
index 000000000..a3a39f025
Binary files /dev/null and b/doc/pub/week40/html/figures/figures/adam.png differ
diff --git a/doc/pub/week40/html/figures/figures/nns.png b/doc/pub/week40/html/figures/figures/nns.png
new file mode 100644
index 000000000..19e31ef05
Binary files /dev/null and b/doc/pub/week40/html/figures/figures/nns.png differ
diff --git a/doc/pub/week40/html/figures/figures/rmsprop.png b/doc/pub/week40/html/figures/figures/rmsprop.png
new file mode 100644
index 000000000..9f336d033
Binary files /dev/null and b/doc/pub/week40/html/figures/figures/rmsprop.png differ
diff --git a/doc/pub/week40/ipynb/figures/figures/adagrad.png b/doc/pub/week40/ipynb/figures/figures/adagrad.png
new file mode 100644
index 000000000..97a9cf908
Binary files /dev/null and b/doc/pub/week40/ipynb/figures/figures/adagrad.png differ
diff --git a/doc/pub/week40/ipynb/figures/figures/adam.png b/doc/pub/week40/ipynb/figures/figures/adam.png
new file mode 100644
index 000000000..a3a39f025
Binary files /dev/null and b/doc/pub/week40/ipynb/figures/figures/adam.png differ
diff --git a/doc/pub/week40/ipynb/figures/figures/nns.png b/doc/pub/week40/ipynb/figures/figures/nns.png
new file mode 100644
index 000000000..19e31ef05
Binary files /dev/null and b/doc/pub/week40/ipynb/figures/figures/nns.png differ
diff --git a/doc/pub/week40/ipynb/figures/figures/rmsprop.png b/doc/pub/week40/ipynb/figures/figures/rmsprop.png
new file mode 100644
index 000000000..9f336d033
Binary files /dev/null and b/doc/pub/week40/ipynb/figures/figures/rmsprop.png differ
diff --git a/doc/pub/week40/ipynb/week40.ipynb b/doc/pub/week40/ipynb/week40.ipynb
index cf0d66003..0da89492e 100644
--- a/doc/pub/week40/ipynb/week40.ipynb
+++ b/doc/pub/week40/ipynb/week40.ipynb
@@ -3,9 +3,7 @@
   {
    "cell_type": "markdown",
    "id": "4687f345",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
     "doconce format html week40.do.txt --no_mako -->\n",
@@ -15,9 +13,7 @@
   {
    "cell_type": "markdown",
    "id": "3da2749d",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "# Week 40: Gradient descent methods (continued) and start Neural networks\n",
     "**Morten Hjorth-Jensen**, Department of Physics, University of Oslo, Norway and Department of Physics and Astronomy and Facility for Rare Ion Beams, Michigan State University, USA\n",
@@ -28,9 +24,7 @@
   {
    "cell_type": "markdown",
    "id": "55819acc",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Plans for week 40"
    ]
@@ -38,9 +32,7 @@
   {
    "cell_type": "markdown",
    "id": "e1252272",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Lecture Monday September 30, 2024\n",
     "1. Stochastic Gradient descent with examples and automatic differentiation\n",
@@ -53,9 +45,7 @@
   {
    "cell_type": "markdown",
    "id": "5684fcf4",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Suggested readings and videos\n",
     "**Readings and Videos:**\n",
@@ -78,9 +68,7 @@
   {
    "cell_type": "markdown",
    "id": "db5aaeaa",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Lab sessions Tuesday and Wednesday\n",
     "**Material for the active learning sessions on Tuesday and Wednesday.**\n",
@@ -97,9 +85,7 @@
   {
    "cell_type": "markdown",
    "id": "358e1ba3",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Summary from last week, using gradient descent methods, limitations\n",
     "\n",
@@ -119,9 +105,7 @@
   {
    "cell_type": "markdown",
    "id": "5f980026",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Simple implementation of GD for OLS, Ridge and Lasso\n",
     "\n",
@@ -133,11 +117,27 @@
    "cell_type": "code",
    "execution_count": 1,
    "id": "1e1a0f93",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
-   "outputs": [],
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Parameters for OLS using gradient descent\n",
+      "[[3.3082004 ]\n",
+      " [4.71564952]\n",
+      " [4.19643278]]\n",
+      "Parameters for Ridge using gradient descent\n",
+      "[[3.87100077]\n",
+      " [3.28108241]\n",
+      " [4.85556155]]\n",
+      "Parameters for Lasso using gradient descent\n",
+      "[[3.36052886]\n",
+      " [4.56274245]\n",
+      " [4.2705634 ]]\n"
+     ]
+    }
+   ],
    "source": [
     "from random import random, seed\n",
     "import numpy as np\n",
@@ -187,9 +187,7 @@
   {
    "cell_type": "markdown",
    "id": "fa35be2d",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## But none of these can compete with Newton's method\n",
     "\n",
@@ -200,11 +198,28 @@
    "cell_type": "code",
    "execution_count": 2,
    "id": "a07b46f9",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
-   "outputs": [],
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Own inversion\n",
+      "[[4.]\n",
+      " [3.]\n",
+      " [5.]]\n",
+      "0 [-26.4958444] [-31.6102394]\n",
+      "1 [-5.24202903e-14] [-5.76114373e-14]\n",
+      "2 [-1.17239551e-15] [-1.43203537e-15]\n",
+      "3 [-1.17239551e-15] [-1.43203537e-15]\n",
+      "4 [-1.17239551e-15] [-1.43203537e-15]\n",
+      "beta from own Newton code\n",
+      "[[4.]\n",
+      " [3.]\n",
+      " [5.]]\n"
+     ]
+    }
+   ],
    "source": [
     "# Using Newton's method\n",
     "from random import random, seed\n",
@@ -244,9 +259,7 @@
   {
    "cell_type": "markdown",
    "id": "ad8a1ec1",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Gradient descent and Logistic regression\n",
     "\n",
@@ -259,11 +272,16 @@
    "cell_type": "code",
    "execution_count": 3,
    "id": "442155a9",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
-   "outputs": [],
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Predictions: [1, 1, 1, 1]\n"
+     ]
+    }
+   ],
    "source": [
     "import numpy as np\n",
     "class LogisticRegression:\n",
@@ -301,9 +319,7 @@
   {
    "cell_type": "markdown",
    "id": "6906a7b0",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Overview video on Stochastic Gradient Descent\n",
     "\n",
@@ -320,9 +336,7 @@
   {
    "cell_type": "markdown",
    "id": "0d1b86f0",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Batches and mini-batches\n",
     "\n",
@@ -341,9 +355,7 @@
   {
    "cell_type": "markdown",
    "id": "de264c3b",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Stochastic Gradient Descent (SGD)\n",
     "\n",
@@ -373,9 +385,7 @@
   {
    "cell_type": "markdown",
    "id": "89be850a",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Stochastic Gradient Descent\n",
     "\n",
@@ -390,9 +400,7 @@
   {
    "cell_type": "markdown",
    "id": "97f3f2d2",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$$\n",
     "C(\\mathbf{\\beta}) = \\sum_{i=1}^n c_i(\\mathbf{x}_i,\n",
@@ -403,9 +411,7 @@
   {
    "cell_type": "markdown",
    "id": "c5756174",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Computation of gradients\n",
     "\n",
@@ -416,9 +422,7 @@
   {
    "cell_type": "markdown",
    "id": "9491611c",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$$\n",
     "\\nabla_\\beta C(\\mathbf{\\beta}) = \\sum_i^n \\nabla_\\beta c_i(\\mathbf{x}_i,\n",
@@ -429,9 +433,7 @@
   {
    "cell_type": "markdown",
    "id": "c5aa3260",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Stochasticity/randomness is introduced by only taking the\n",
     "gradient on a subset of the data called minibatches.  If there are $n$\n",
@@ -443,9 +445,7 @@
   {
    "cell_type": "markdown",
    "id": "14db2504",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## SGD example\n",
     "As an example, suppose we have $10$ data points $(\\mathbf{x}_1,\\cdots, \\mathbf{x}_{10})$ \n",
@@ -465,9 +465,7 @@
   {
    "cell_type": "markdown",
    "id": "7246951f",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$$\n",
     "\\nabla_{\\beta}\n",
@@ -480,9 +478,7 @@
   {
    "cell_type": "markdown",
    "id": "1662a436",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## The gradient step\n",
     "\n",
@@ -492,9 +488,7 @@
   {
    "cell_type": "markdown",
    "id": "af377a66",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$$\n",
     "\\beta_{j+1} = \\beta_j - \\gamma_j \\sum_{i \\in B_k}^n \\nabla_\\beta c_i(\\mathbf{x}_i,\n",
@@ -505,9 +499,7 @@
   {
    "cell_type": "markdown",
    "id": "87fd2445",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "where $k$ is picked at random with equal\n",
     "probability from $[1,n/M]$. An iteration over the number of\n",
@@ -519,9 +511,7 @@
   {
    "cell_type": "markdown",
    "id": "e962547b",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Simple example code"
    ]
@@ -530,10 +520,7 @@
    "cell_type": "code",
    "execution_count": 4,
    "id": "53e4501a",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "import numpy as np \n",
@@ -555,9 +542,7 @@
   {
    "cell_type": "markdown",
    "id": "46109bad",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Taking the gradient only on a subset of the data has two important\n",
     "benefits. First, it introduces randomness which decreases the chance\n",
@@ -571,9 +556,7 @@
   {
    "cell_type": "markdown",
    "id": "539812ca",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## When do we stop?\n",
     "\n",
@@ -592,9 +575,7 @@
   {
    "cell_type": "markdown",
    "id": "bdfb3ce0",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Slightly different approach\n",
     "\n",
@@ -612,9 +593,7 @@
   {
    "cell_type": "markdown",
    "id": "93825012",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Time decay rate\n",
     "\n",
@@ -631,10 +610,7 @@
    "cell_type": "code",
    "execution_count": 5,
    "id": "17635e19",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "import numpy as np \n",
@@ -666,9 +642,7 @@
   {
    "cell_type": "markdown",
    "id": "e9af4782",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Code with a Number of Minibatches which varies\n",
     "\n",
@@ -679,10 +653,7 @@
    "cell_type": "code",
    "execution_count": 6,
    "id": "a21b3bf0",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "%matplotlib inline\n",
@@ -758,9 +729,7 @@
   {
    "cell_type": "markdown",
    "id": "5649ec2a",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Replace or not\n",
     "\n",
@@ -773,9 +742,7 @@
   {
    "cell_type": "markdown",
    "id": "302f4f77",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Momentum based GD\n",
     "\n",
@@ -788,9 +755,7 @@
   {
    "cell_type": "markdown",
    "id": "0c094b0b",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$$\n",
     "\\mathbf{v}_{t}=\\gamma \\mathbf{v}_{t-1}+\\eta_{t}\\nabla_\\theta E(\\boldsymbol{\\theta}_t) \\nonumber\n",
@@ -800,9 +765,7 @@
   {
    "cell_type": "markdown",
    "id": "750579fd",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "<!-- Equation labels as ordinary links -->\n",
     "<div id=\"_auto1\"></div>\n",
@@ -818,9 +781,7 @@
   {
    "cell_type": "markdown",
    "id": "1d55fdab",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "where we have introduced a momentum parameter $\\gamma$, with\n",
     "$0\\le\\gamma\\le 1$, and for brevity we dropped the explicit notation to\n",
@@ -837,9 +798,7 @@
   {
    "cell_type": "markdown",
    "id": "c78ccbf6",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$$\n",
     "\\Delta \\boldsymbol{\\theta}_{t+1} = \\gamma \\Delta \\boldsymbol{\\theta}_t -\\ \\eta_{t}\\nabla_\\theta E(\\boldsymbol{\\theta}_t),\n",
@@ -849,9 +808,7 @@
   {
    "cell_type": "markdown",
    "id": "73bf75c1",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "where we have defined $\\Delta \\boldsymbol{\\theta}_{t}= \\boldsymbol{\\theta}_t-\\boldsymbol{\\theta}_{t-1}$."
    ]
@@ -859,9 +816,7 @@
   {
    "cell_type": "markdown",
    "id": "24406ec7",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## More on momentum based approaches\n",
     "\n",
@@ -875,9 +830,7 @@
   {
    "cell_type": "markdown",
    "id": "ff0a7cdb",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$$\n",
     "m {d^2 \\mathbf{w} \\over dt^2} + \\mu {d \\mathbf{w} \\over dt }= -\\nabla_w E(\\mathbf{w}).\n",
@@ -887,9 +840,7 @@
   {
    "cell_type": "markdown",
    "id": "5881f048",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "We can discretize this equation in the usual way to get"
    ]
@@ -897,9 +848,7 @@
   {
    "cell_type": "markdown",
    "id": "ceb5d976",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$$\n",
     "m { \\mathbf{w}_{t+\\Delta t}-2 \\mathbf{w}_{t} +\\mathbf{w}_{t-\\Delta t} \\over (\\Delta t)^2}+\\mu {\\mathbf{w}_{t+\\Delta t}- \\mathbf{w}_{t} \\over \\Delta t} = -\\nabla_w E(\\mathbf{w}).\n",
@@ -909,9 +858,7 @@
   {
    "cell_type": "markdown",
    "id": "4e997304",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Rearranging this equation, we can rewrite this as"
    ]
@@ -919,9 +866,7 @@
   {
    "cell_type": "markdown",
    "id": "db031583",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$$\n",
     "\\Delta \\mathbf{w}_{t +\\Delta t}= - { (\\Delta t)^2 \\over m +\\mu \\Delta t} \\nabla_w E(\\mathbf{w})+ {m \\over m +\\mu \\Delta t} \\Delta \\mathbf{w}_t.\n",
@@ -931,9 +876,7 @@
   {
    "cell_type": "markdown",
    "id": "cf54750e",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Momentum parameter\n",
     "\n",
@@ -947,9 +890,7 @@
   {
    "cell_type": "markdown",
    "id": "6424326d",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$$\n",
     "\\gamma= {m \\over m +\\mu \\Delta t }, \\qquad \\eta = {(\\Delta t)^2 \\over m +\\mu \\Delta t}.\n",
@@ -959,9 +900,7 @@
   {
    "cell_type": "markdown",
    "id": "4c52261d",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Thus, as the name suggests, the momentum parameter is proportional to\n",
     "the mass of the particle and effectively provides inertia.\n",
@@ -992,9 +931,7 @@
   {
    "cell_type": "markdown",
    "id": "e6351f61",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$$\n",
     "\\mathbf{v}_{t}=\\gamma \\mathbf{v}_{t-1}+\\eta_{t}\\nabla_\\theta E(\\boldsymbol{\\theta}_t +\\gamma \\mathbf{v}_{t-1}) \\nonumber\n",
@@ -1004,9 +941,7 @@
   {
    "cell_type": "markdown",
    "id": "c0feb3d3",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "<!-- Equation labels as ordinary links -->\n",
     "<div id=\"_auto2\"></div>\n",
@@ -1022,9 +957,7 @@
   {
    "cell_type": "markdown",
    "id": "932dfc89",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "One of the major advantages of NAG is that it allows for the use of a larger learning rate than GDM for the same choice of $\\gamma$."
    ]
@@ -1032,9 +965,7 @@
   {
    "cell_type": "markdown",
    "id": "d403c5d0",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Second moment of the gradient\n",
     "\n",
@@ -1063,9 +994,7 @@
   {
    "cell_type": "markdown",
    "id": "3ecb97e6",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## RMS prop\n",
     "\n",
@@ -1078,9 +1007,7 @@
   {
    "cell_type": "markdown",
    "id": "fabedf61",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "<!-- Equation labels as ordinary links -->\n",
     "<div id=\"_auto3\"></div>\n",
@@ -1096,9 +1023,7 @@
   {
    "cell_type": "markdown",
    "id": "e0a1b163",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$$\n",
     "\\mathbf{s}_t =\\beta \\mathbf{s}_{t-1} +(1-\\beta)\\mathbf{g}_t^2 \\nonumber\n",
@@ -1108,9 +1033,7 @@
   {
    "cell_type": "markdown",
    "id": "60639b97",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$$\n",
     "\\boldsymbol{\\theta}_{t+1}=\\boldsymbol{\\theta}_t - \\eta_t { \\mathbf{g}_t \\over \\sqrt{\\mathbf{s}_t +\\epsilon}}, \\nonumber\n",
@@ -1120,9 +1043,7 @@
   {
    "cell_type": "markdown",
    "id": "f3af4ae4",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "where $\\beta$ controls the averaging time of the second moment and is\n",
     "typically taken to be about $\\beta=0.9$, $\\eta_t$ is a learning rate\n",
@@ -1138,9 +1059,7 @@
   {
    "cell_type": "markdown",
    "id": "936f4678",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## [ADAM optimizer](https://arxiv.org/abs/1412.6980)\n",
     "\n",
@@ -1167,9 +1086,7 @@
   {
    "cell_type": "markdown",
    "id": "9614f73b",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "<!-- Equation labels as ordinary links -->\n",
     "<div id=\"_auto4\"></div>\n",
@@ -1185,9 +1102,7 @@
   {
    "cell_type": "markdown",
    "id": "2440882c",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$$\n",
     "\\mathbf{m}_t = \\beta_1 \\mathbf{m}_{t-1} + (1-\\beta_1) \\mathbf{g}_t \\nonumber\n",
@@ -1197,9 +1112,7 @@
   {
    "cell_type": "markdown",
    "id": "a585a1e4",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$$\n",
     "\\mathbf{s}_t =\\beta_2 \\mathbf{s}_{t-1} +(1-\\beta_2)\\mathbf{g}_t^2 \\nonumber\n",
@@ -1209,9 +1122,7 @@
   {
    "cell_type": "markdown",
    "id": "c6a8e863",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$$\n",
     "\\boldsymbol{\\mathbf{m}}_t={\\mathbf{m}_t \\over 1-\\beta_1^t} \\nonumber\n",
@@ -1221,9 +1132,7 @@
   {
    "cell_type": "markdown",
    "id": "1978dbb2",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$$\n",
     "\\boldsymbol{\\mathbf{s}}_t ={\\mathbf{s}_t \\over1-\\beta_2^t} \\nonumber\n",
@@ -1233,9 +1142,7 @@
   {
    "cell_type": "markdown",
    "id": "0b153be5",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$$\n",
     "\\boldsymbol{\\theta}_{t+1}=\\boldsymbol{\\theta}_t - \\eta_t { \\boldsymbol{\\mathbf{m}}_t \\over \\sqrt{\\boldsymbol{\\mathbf{s}}_t} +\\epsilon}, \\nonumber\n",
@@ -1245,9 +1152,7 @@
   {
    "cell_type": "markdown",
    "id": "2659e6e6",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "<!-- Equation labels as ordinary links -->\n",
     "<div id=\"_auto5\"></div>\n",
@@ -1262,9 +1167,7 @@
   {
    "cell_type": "markdown",
    "id": "fe39f4bd",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "where $\\beta_1$ and $\\beta_2$ set the memory lifetime of the first and\n",
     "second moment and are typically taken to be $0.9$ and $0.99$\n",
@@ -1281,9 +1184,7 @@
   {
    "cell_type": "markdown",
    "id": "45c3b53e",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$$\n",
     "\\Delta \\theta_{t+1}= -\\eta_t { \\boldsymbol{m}_t \\over \\sqrt{\\sigma_t^2 +  m_t^2 }+\\epsilon}.\n",
@@ -1293,9 +1194,7 @@
   {
    "cell_type": "markdown",
    "id": "0e1d1ab6",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Algorithms and codes for Adagrad, RMSprop and Adam\n",
     "\n",
@@ -1307,9 +1206,7 @@
   {
    "cell_type": "markdown",
    "id": "650e1cd6",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## AdaGrad algorithm, taken from [Goodfellow et al](https://www.deeplearningbook.org/contents/optimization.html)\n",
     "\n",
@@ -1323,9 +1220,7 @@
   {
    "cell_type": "markdown",
    "id": "36fcf928",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## RMSProp algorithm, taken from [Goodfellow et al](https://www.deeplearningbook.org/contents/optimization.html)\n",
     "\n",
@@ -1339,9 +1234,7 @@
   {
    "cell_type": "markdown",
    "id": "f68ae4d9",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## ADAM algorithm, taken from [Goodfellow et al](https://www.deeplearningbook.org/contents/optimization.html)\n",
     "\n",
@@ -1355,9 +1248,7 @@
   {
    "cell_type": "markdown",
    "id": "8eb914ed",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Practical tips\n",
     "\n",
@@ -1375,9 +1266,7 @@
   {
    "cell_type": "markdown",
    "id": "5e2981dc",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Automatic differentiation\n",
     "\n",
@@ -1413,9 +1302,7 @@
   {
    "cell_type": "markdown",
    "id": "b81eabd7",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$$\n",
     "f(x) = \\sin\\left(2\\pi x + x^2\\right)\n",
@@ -1425,9 +1312,7 @@
   {
    "cell_type": "markdown",
    "id": "664845f7",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "which has the following derivative"
    ]
@@ -1435,9 +1320,7 @@
   {
    "cell_type": "markdown",
    "id": "a2a77f69",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$$\n",
     "f'(x) = \\cos\\left(2\\pi x + x^2\\right)\\left(2\\pi + 2x\\right)\n",
@@ -1447,9 +1330,7 @@
   {
    "cell_type": "markdown",
    "id": "34f3786a",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Using **autograd** we have"
    ]
@@ -1458,10 +1339,7 @@
    "cell_type": "code",
    "execution_count": 7,
    "id": "6f76195d",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "import autograd.numpy as np\n",
@@ -1503,9 +1381,7 @@
   {
    "cell_type": "markdown",
    "id": "d2d23a0d",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Using autograd\n",
     "\n",
@@ -1520,10 +1396,7 @@
    "cell_type": "code",
    "execution_count": 8,
    "id": "c51dc30f",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "import autograd.numpy as np\n",
@@ -1548,9 +1421,7 @@
   {
    "cell_type": "markdown",
    "id": "c0b62f5e",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Autograd with more complicated functions\n",
     "\n",
@@ -1563,10 +1434,7 @@
    "cell_type": "code",
    "execution_count": 9,
    "id": "52904dfe",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "import autograd.numpy as np\n",
@@ -1607,9 +1475,7 @@
   {
    "cell_type": "markdown",
    "id": "86e0ebe1",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Note that the grad function will not produce the true gradient of the function. The true gradient of a function with two or more variables will produce a vector, where each element is the function differentiated w.r.t a variable."
    ]
@@ -1617,9 +1483,7 @@
   {
    "cell_type": "markdown",
    "id": "634ceac6",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## More complicated functions using the elements of their arguments directly"
    ]
@@ -1628,10 +1492,7 @@
    "cell_type": "code",
    "execution_count": 10,
    "id": "2bea8a29",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "import autograd.numpy as np\n",
@@ -1656,9 +1517,7 @@
   {
    "cell_type": "markdown",
    "id": "6275cbc4",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Note that in this case, when sending an array as input argument, the\n",
     "output from Autograd is another array. This is the true gradient of\n",
@@ -1671,9 +1530,7 @@
   {
    "cell_type": "markdown",
    "id": "0068cd3c",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Functions using mathematical functions from Numpy"
    ]
@@ -1682,10 +1539,7 @@
    "cell_type": "code",
    "execution_count": 11,
    "id": "efa2537a",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "import autograd.numpy as np\n",
@@ -1710,9 +1564,7 @@
   {
    "cell_type": "markdown",
    "id": "0e26d1c9",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## More autograd"
    ]
@@ -1721,10 +1573,7 @@
    "cell_type": "code",
    "execution_count": 12,
    "id": "a5630d31",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "import autograd.numpy as np\n",
@@ -1746,9 +1595,7 @@
   {
    "cell_type": "markdown",
    "id": "b110f814",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## And  with loops"
    ]
@@ -1757,10 +1604,7 @@
    "cell_type": "code",
    "execution_count": 13,
    "id": "a6ebc226",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "import autograd.numpy as np\n",
@@ -1793,10 +1637,7 @@
    "cell_type": "code",
    "execution_count": 14,
    "id": "c38d8cd5",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "import autograd.numpy as np\n",
@@ -1813,9 +1654,7 @@
   {
    "cell_type": "markdown",
    "id": "a85b9833",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Using recursion"
    ]
@@ -1824,10 +1663,7 @@
    "cell_type": "code",
    "execution_count": 15,
    "id": "cc00d05b",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "import autograd.numpy as np\n",
@@ -1862,9 +1698,7 @@
   {
    "cell_type": "markdown",
    "id": "2b8de443",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Note that if n is equal to zero or one, Autograd will give an error message. This message appears when the output is independent on input."
    ]
@@ -1872,9 +1706,7 @@
   {
    "cell_type": "markdown",
    "id": "7cd545c2",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Using Autograd with OLS\n",
     "\n",
@@ -1887,10 +1719,7 @@
    "cell_type": "code",
    "execution_count": 16,
    "id": "4bd975ee",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "# Using Autograd to calculate gradients for OLS\n",
@@ -1947,9 +1776,7 @@
   {
    "cell_type": "markdown",
    "id": "f442d7c8",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Same code but now with momentum gradient descent"
    ]
@@ -1958,10 +1785,7 @@
    "cell_type": "code",
    "execution_count": 17,
    "id": "26b37007",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "# Using Autograd to calculate gradients for OLS\n",
@@ -2022,9 +1846,7 @@
   {
    "cell_type": "markdown",
    "id": "acb26c62",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Including Stochastic Gradient Descent with Autograd\n",
     "In this code we include the stochastic gradient descent approach discussed above. Note here that we specify which argument we are taking the derivative with respect to when using **autograd**."
@@ -2034,10 +1856,7 @@
    "cell_type": "code",
    "execution_count": 18,
    "id": "9d6704a9",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "# Using Autograd to calculate gradients using SGD\n",
@@ -2118,9 +1937,7 @@
   {
    "cell_type": "markdown",
    "id": "dd2d9f35",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Same code but now with momentum gradient descent"
    ]
@@ -2129,10 +1946,7 @@
    "cell_type": "code",
    "execution_count": 19,
    "id": "85731b81",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "# Using Autograd to calculate gradients using SGD\n",
@@ -2207,9 +2021,7 @@
   {
    "cell_type": "markdown",
    "id": "99fcedcf",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Similar (second order function now) problem but now with AdaGrad"
    ]
@@ -2218,10 +2030,7 @@
    "cell_type": "code",
    "execution_count": 20,
    "id": "1bbef9b5",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "# Using Autograd to calculate gradients using AdaGrad and Stochastic Gradient descent\n",
@@ -2277,9 +2086,7 @@
   {
    "cell_type": "markdown",
    "id": "6f14b9e9",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Running this code we note an almost perfect agreement with the results from matrix inversion."
    ]
@@ -2287,9 +2094,7 @@
   {
    "cell_type": "markdown",
    "id": "420764bb",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## RMSprop for adaptive learning rate with Stochastic Gradient Descent"
    ]
@@ -2298,10 +2103,7 @@
    "cell_type": "code",
    "execution_count": 21,
    "id": "1ea8deac",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "# Using Autograd to calculate gradients using RMSprop  and Stochastic Gradient descent\n",
@@ -2363,9 +2165,7 @@
   {
    "cell_type": "markdown",
    "id": "7476e8fa",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## And finally [ADAM](https://arxiv.org/pdf/1412.6980.pdf)"
    ]
@@ -2374,10 +2174,7 @@
    "cell_type": "code",
    "execution_count": 22,
    "id": "dd54ee0d",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "# Using Autograd to calculate gradients using RMSprop  and Stochastic Gradient descent\n",
@@ -2444,9 +2241,7 @@
   {
    "cell_type": "markdown",
    "id": "8bff90bf",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## And Logistic Regression"
    ]
@@ -2455,10 +2250,7 @@
    "cell_type": "code",
    "execution_count": 23,
    "id": "4d6f38aa",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "import autograd.numpy as np\n",
@@ -2499,9 +2291,7 @@
   {
    "cell_type": "markdown",
    "id": "be009d46",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Introducing [JAX](https://jax.readthedocs.io/en/latest/)\n",
     "\n",
@@ -2515,9 +2305,7 @@
   {
    "cell_type": "markdown",
    "id": "8c42305d",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "### Getting started with Jax, note the way we import numpy"
    ]
@@ -2526,10 +2314,7 @@
    "cell_type": "code",
    "execution_count": 24,
    "id": "ee92fe44",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "import jax\n",
@@ -2543,9 +2328,7 @@
   {
    "cell_type": "markdown",
    "id": "7b2c5c5d",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "### A warm-up example"
    ]
@@ -2554,10 +2337,7 @@
    "cell_type": "code",
    "execution_count": 25,
    "id": "a9f25005",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "def function(x):\n",
@@ -2599,9 +2379,7 @@
   {
    "cell_type": "markdown",
    "id": "6734e909",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "### A more advanced example"
    ]
@@ -2610,10 +2388,7 @@
    "cell_type": "code",
    "execution_count": 26,
    "id": "cc5b54ed",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "backend = np\n",
@@ -2641,9 +2416,7 @@
   {
    "cell_type": "markdown",
    "id": "d0ff0a0e",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Introduction to Neural networks\n",
     "\n",
@@ -2659,9 +2432,7 @@
   {
    "cell_type": "markdown",
    "id": "624b7343",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Artificial neurons\n",
     "\n",
@@ -2683,9 +2454,7 @@
   {
    "cell_type": "markdown",
    "id": "b6ab860b",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "<!-- Equation labels as ordinary links -->\n",
     "<div id=\"artificialNeuron\"></div>\n",
@@ -2701,9 +2470,7 @@
   {
    "cell_type": "markdown",
    "id": "fdab69ac",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Here, the output $y$ of the neuron is the value of its activation function, which have as input\n",
     "a weighted sum of signals $x_i, \\dots ,x_n$ received by $n$ other neurons.\n",
@@ -2741,9 +2508,7 @@
   {
    "cell_type": "markdown",
    "id": "6466c987",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Neural network types\n",
     "\n",
@@ -2770,9 +2535,7 @@
   {
    "cell_type": "markdown",
    "id": "8a9c8edb",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Feed-forward neural networks\n",
     "\n",
@@ -2791,9 +2554,7 @@
   {
    "cell_type": "markdown",
    "id": "ca6a908c",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Convolutional Neural Network\n",
     "\n",
@@ -2820,9 +2581,7 @@
   {
    "cell_type": "markdown",
    "id": "7f1f4f38",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Recurrent neural networks\n",
     "\n",
@@ -2841,9 +2600,7 @@
   {
    "cell_type": "markdown",
    "id": "ced07fa5",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Other types of networks\n",
     "\n",
@@ -2862,9 +2619,7 @@
   {
    "cell_type": "markdown",
    "id": "5421341a",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Multilayer perceptrons\n",
     "\n",
@@ -2879,9 +2634,7 @@
   {
    "cell_type": "markdown",
    "id": "e5ae78f7",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Why multilayer perceptrons?\n",
     "\n",
@@ -2900,9 +2653,7 @@
   {
    "cell_type": "markdown",
    "id": "40adefcc",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Illustration of a single perceptron model and a multi-perceptron model\n",
     "\n",
@@ -2916,9 +2667,7 @@
   {
    "cell_type": "markdown",
    "id": "31608ee5",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Examples of XOR, OR and AND gates\n",
     "\n",
@@ -2933,10 +2682,7 @@
    "cell_type": "code",
    "execution_count": 27,
    "id": "0529c3e9",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "\"\"\"\n",
@@ -2974,9 +2720,7 @@
   {
    "cell_type": "markdown",
    "id": "844f16f8",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "What is happening here?"
    ]
@@ -2984,9 +2728,7 @@
   {
    "cell_type": "markdown",
    "id": "750cf959",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Does Logistic Regression do a better Job?"
    ]
@@ -2995,10 +2737,7 @@
    "cell_type": "code",
    "execution_count": 28,
    "id": "68870b7b",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "\"\"\"\n",
@@ -3055,9 +2794,7 @@
   {
    "cell_type": "markdown",
    "id": "78424809",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Not exactly impressive, but somewhat better."
    ]
@@ -3065,9 +2802,7 @@
   {
    "cell_type": "markdown",
    "id": "51a551b4",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Adding Neural Networks"
    ]
@@ -3076,10 +2811,7 @@
    "cell_type": "code",
    "execution_count": 29,
    "id": "390527c4",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "\n",
@@ -3096,9 +2828,7 @@
   {
    "cell_type": "markdown",
    "id": "3fc8196f",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Mathematical model\n",
     "\n",
@@ -3108,9 +2838,7 @@
   {
    "cell_type": "markdown",
    "id": "f75b24d9",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$$\n",
     "y = f\\left(\\sum_{i=1}^n w_ix_i + b_i\\right) = f(z),\n",
@@ -3120,9 +2848,7 @@
   {
    "cell_type": "markdown",
    "id": "3a468c58",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "This function receives $x_i$ as inputs.\n",
     "Here the activation $z=(\\sum_{i=1}^n w_ix_i+b_i)$. \n",
@@ -3135,9 +2861,7 @@
   {
    "cell_type": "markdown",
    "id": "bf8973da",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Mathematical model\n",
     "\n",
@@ -3147,9 +2871,7 @@
   {
    "cell_type": "markdown",
    "id": "5e0da5ad",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "<!-- Equation labels as ordinary links -->\n",
     "<div id=\"_auto6\"></div>\n",
@@ -3164,9 +2886,7 @@
   {
    "cell_type": "markdown",
    "id": "64d27300",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Here $b_i$ is the so-called bias which is normally needed in\n",
     "case of zero activation weights or inputs. How to fix the biases and\n",
@@ -3179,9 +2899,7 @@
   {
    "cell_type": "markdown",
    "id": "9e29b76e",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "<!-- Equation labels as ordinary links -->\n",
     "<div id=\"outputLayer1\"></div>\n",
@@ -3197,9 +2915,7 @@
   {
    "cell_type": "markdown",
    "id": "b1da5364",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "where we assume that all nodes in the same layer have identical\n",
     "activation functions, hence the notation $f$. In general, we could assume in the more general case that different layers have different activation functions.\n",
@@ -3209,9 +2925,7 @@
   {
    "cell_type": "markdown",
    "id": "6ed696f9",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "<!-- Equation labels as ordinary links -->\n",
     "<div id=\"generalLayer\"></div>\n",
@@ -3227,9 +2941,7 @@
   {
    "cell_type": "markdown",
    "id": "a4148a64",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "where $N_l$ is the number of nodes in layer $l$. When the output of\n",
     "all the nodes in the first hidden layer are computed, the values of\n",
@@ -3240,9 +2952,7 @@
   {
    "cell_type": "markdown",
    "id": "da587faa",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Mathematical model\n",
     "\n",
@@ -3252,9 +2962,7 @@
   {
    "cell_type": "markdown",
    "id": "73eefe7b",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "<!-- Equation labels as ordinary links -->\n",
     "<div id=\"_auto7\"></div>\n",
@@ -3270,9 +2978,7 @@
   {
    "cell_type": "markdown",
    "id": "9c00114f",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "<!-- Equation labels as ordinary links -->\n",
     "<div id=\"outputLayer2\"></div>\n",
@@ -3288,9 +2994,7 @@
   {
    "cell_type": "markdown",
    "id": "5e51f9a8",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "where we have substituted $y_k^1$ with the inputs $x_k$. Finally, the ANN output reads"
    ]
@@ -3298,9 +3002,7 @@
   {
    "cell_type": "markdown",
    "id": "f0e1bc9b",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "<!-- Equation labels as ordinary links -->\n",
     "<div id=\"_auto8\"></div>\n",
@@ -3316,9 +3018,7 @@
   {
    "cell_type": "markdown",
    "id": "992a154d",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "<!-- Equation labels as ordinary links -->\n",
     "<div id=\"_auto9\"></div>\n",
@@ -3335,9 +3035,7 @@
   {
    "cell_type": "markdown",
    "id": "1ce03e37",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Mathematical model\n",
     "\n",
@@ -3348,9 +3046,7 @@
   {
    "cell_type": "markdown",
    "id": "1d14ed5f",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "<!-- Equation labels as ordinary links -->\n",
     "<div id=\"completeNN\"></div>\n",
@@ -3366,9 +3062,7 @@
   {
    "cell_type": "markdown",
    "id": "5d4af831",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "which illustrates a basic property of MLPs: The only independent\n",
     "variables are the input values $x_n$."
@@ -3377,9 +3071,7 @@
   {
    "cell_type": "markdown",
    "id": "833c2044",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Mathematical model\n",
     "\n",
@@ -3396,9 +3088,7 @@
   {
    "cell_type": "markdown",
    "id": "1919c37b",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "<!-- Equation labels as ordinary links -->\n",
     "<div id=\"_auto10\"></div>\n",
@@ -3414,9 +3104,7 @@
   {
    "cell_type": "markdown",
    "id": "034185ce",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "where the parameters $c_i$ are weights and biases. By adjusting these\n",
     "parameters, the activation functions can be shifted up and down or\n",
@@ -3427,9 +3115,7 @@
   {
    "cell_type": "markdown",
    "id": "36e298b2",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "### Matrix-vector notation\n",
     "\n",
@@ -3447,9 +3133,7 @@
   {
    "cell_type": "markdown",
    "id": "b8f05873",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "<!-- Equation labels as ordinary links -->\n",
     "<div id=\"_auto11\"></div>\n",
@@ -3480,9 +3164,7 @@
   {
    "cell_type": "markdown",
    "id": "e7a6e970",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "### Matrix-vector notation  and activation\n",
     "\n",
@@ -3492,9 +3174,7 @@
   {
    "cell_type": "markdown",
    "id": "543db865",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "<!-- Equation labels as ordinary links -->\n",
     "<div id=\"_auto12\"></div>\n",
@@ -3511,9 +3191,7 @@
   {
    "cell_type": "markdown",
    "id": "59d15e0e",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "This is not just a convenient and compact notation, but also a useful\n",
     "and intuitive way to think about MLPs: The output is calculated by a\n",
@@ -3525,9 +3203,7 @@
   {
    "cell_type": "markdown",
    "id": "ccf5313b",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "### Activation functions\n",
     "\n",
@@ -3548,9 +3224,7 @@
   {
    "cell_type": "markdown",
    "id": "4123228a",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "### Activation functions, Logistic and Hyperbolic ones\n",
     "\n",
@@ -3567,9 +3241,7 @@
   {
    "cell_type": "markdown",
    "id": "a388e95b",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$$\n",
     "f(x) = \\frac{1}{1 + e^{-x}},\n",
@@ -3579,9 +3251,7 @@
   {
    "cell_type": "markdown",
    "id": "1b615cdb",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "and the *hyperbolic tangent* function"
    ]
@@ -3589,9 +3259,7 @@
   {
    "cell_type": "markdown",
    "id": "89752e18",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$$\n",
     "f(x) = \\tanh(x)\n",
@@ -3601,9 +3269,7 @@
   {
    "cell_type": "markdown",
    "id": "b0006d32",
-   "metadata": {
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "### Relevance\n",
     "\n",
@@ -3618,10 +3284,7 @@
    "cell_type": "code",
    "execution_count": 30,
    "id": "bcba3ef9",
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "\"\"\"The sigmoid function (or the logistic curve) is a \n",
@@ -3698,7 +3361,25 @@
    ]
   }
  ],
- "metadata": {},
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.18"
+  }
+ },
  "nbformat": 4,
  "nbformat_minor": 5
 }