update week 36

doc/LectureNotes/_build/html/_sources/exercisesweek37.ipynb

@@ -2,7 +2,7 @@
  "cells": [
   {
    "cell_type": "markdown",
-   "id": "1b638450",
+   "id": "1b941c35",
    "metadata": {
     "editable": true
    },
@@ -14,7 +14,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "60060788",
+   "id": "dc05b096",
    "metadata": {
     "editable": true
    },
@@ -27,7 +27,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "7cdd88e4",
+   "id": "2cf07405",
    "metadata": {
     "editable": true
    },
@@ -37,7 +37,7 @@
     "After having completed these exercises you will have:\n",
     "1. Your own code for the implementation of the simplest gradient descent approach applied to ordinary least squares (OLS) and Ridge regression\n",
     "\n",
-    "2. Be able to compare the analytical expressions for OLS and Rudge regression with the gradient descent approach\n",
+    "2. Be able to compare the analytical expressions for OLS and Ridge regression with the gradient descent approach\n",
     "\n",
     "3. Explore the role of the learning rate in the gradient descent approach and the hyperparameter $\\lambda$ in Ridge regression\n",
     "\n",
@@ -46,7 +46,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "0328fa2a",
+   "id": "3c139edb",
    "metadata": {
     "editable": true
    },
@@ -58,7 +58,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "ac760265",
+   "id": "aad4cfac",
    "metadata": {
     "editable": true
    },
@@ -70,7 +70,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "8d4b0753",
+   "id": "6682282f",
    "metadata": {
     "editable": true
    },
@@ -83,7 +83,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "4517d311",
+   "id": "89e2f4c4",
    "metadata": {
     "editable": true
    },
@@ -99,7 +99,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "da1834a9",
+   "id": "b06d4e53",
    "metadata": {
     "editable": true
    },
@@ -120,7 +120,7 @@
   {
    "cell_type": "code",
    "execution_count": 1,
-   "id": "590b2fb0",
+   "id": "63796480",
    "metadata": {
     "collapsed": false,
     "editable": true
@@ -140,7 +140,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "24dd92fc",
+   "id": "80748600",
    "metadata": {
     "editable": true
    },
@@ -156,7 +156,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "4ba80e16",
+   "id": "92751e5f",
    "metadata": {
     "editable": true
    },
@@ -168,18 +168,18 @@
   },
   {
    "cell_type": "markdown",
-   "id": "be65f56f",
+   "id": "aedfbd7a",
    "metadata": {
     "editable": true
    },
    "source": [
-    "## Exercise 3, use the analytical formulae for OLS and Ridge regression to find the optimal paramters $\\boldsymbol{\\theta}$"
+    "## Exercise 3, using the analytical formulae for OLS and Ridge regression to find the optimal paramters $\\boldsymbol{\\theta}$"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 2,
-   "id": "c265677e",
+   "id": "5d1288fa",
    "metadata": {
     "collapsed": false,
     "editable": true
@@ -200,7 +200,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "b989efb9",
+   "id": "628f5e89",
    "metadata": {
     "editable": true
    },
@@ -214,7 +214,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "17b08af7",
+   "id": "f115ba4e",
    "metadata": {
     "editable": true
    },
@@ -226,7 +226,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "6acd4708",
+   "id": "a9b5189c",
    "metadata": {
     "editable": true
    },
@@ -238,7 +238,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "50579422",
+   "id": "a3969ff6",
    "metadata": {
     "editable": true
    },
@@ -258,7 +258,7 @@
   {
    "cell_type": "code",
    "execution_count": 3,
-   "id": "c57cc917",
+   "id": "34d87303",
    "metadata": {
     "collapsed": false,
     "editable": true
@@ -277,13 +277,14 @@
     "cost_history = np.zeros(num_iters)\n",
     "\n",
     "# Gradient descent loop\n",
-    "m = n_samples  # number of examples\n",
+    "m = n_samples  # number of data points\n",
     "for t in range(num_iters):\n",
     "    # Compute prediction error\n",
     "    error = X_norm.dot(theta) - y_centered \n",
     "    # Compute cost for OLS and Ridge (MSE + regularization for Ridge) for monitoring\n",
     "    cost_OLS = ?\n",
     "    cost_Ridge = ?\n",
+    "    # You could add a history for both methods (optional)\n",
     "    cost_history[t] = ?\n",
     "    # Compute gradients for OSL and Ridge\n",
     "    grad_OLS = ?\n",
@@ -301,7 +302,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "e2654903",
+   "id": "989f70bb",
    "metadata": {
     "editable": true
    },
@@ -313,19 +314,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "7e7e7de6",
+   "id": "370b2dad",
    "metadata": {
     "editable": true
    },
    "source": [
     "### 4b)\n",
     "\n",
-    "Try to add a stopping parameter as function of the number iterations. How would you define a stopping criterion?"
+    "Try to add a stopping parameter as function of the number iterations and the difference between the new and old $\\theta$ values. How would you define a stopping criterion?"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "75542708",
+   "id": "ef197cd7",
    "metadata": {
     "editable": true
    },
@@ -351,7 +352,7 @@
   {
    "cell_type": "code",
    "execution_count": 4,
-   "id": "06077986",
+   "id": "4ccc2f65",
    "metadata": {
     "collapsed": false,
     "editable": true
@@ -380,7 +381,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "86d46505",
+   "id": "00e279ef",
    "metadata": {
     "editable": true
    },
@@ -394,7 +395,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "f1d2f848",
+   "id": "c910b3f4",
    "metadata": {
     "editable": true
    },
@@ -406,7 +407,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "f1c91e84",
+   "id": "89e6e040",
    "metadata": {
     "editable": true
    },