update on book

doc/BookChapters/chapter3.do.txt

@@ -1245,6 +1245,7 @@ y_pred = y_pred + y_train_mean
 !ec
 
 
+
 Let us try to understand what this may imply mathematically when we
 subtract the mean values, also known as *zero centering*. For
 simplicity, we will focus on  ordinary regression, as done in the above example.
@@ -1281,20 +1282,6 @@ Multiplying away the constant $2/n$, we obtain
 \]
 !et
 
-
-We assume 
-that every column of $\bm{X}$ is centered, which we can do by subtracting the mean,
-!bc pycod
-X = X - np.mean(X,axis=0)
-!ec
-
-This means that we need to rewrite $X_{ij}$ as $\tilde{X}_{ij}=X_{ij}-\mu_j$, where
-!bt
-\[
-\mu_j = \frac{1}{n}\sum_{i=0}^{n-1}X_{ij}.
-\]
-!et
-
 Let us special first to the case where we have only two parameters $\beta_0$ and $\beta_1$.
 Our result for $\beta_0$ simplifies then to
 !bt
@@ -1302,13 +1289,13 @@ Our result for $\beta_0$ simplifies then to
 n\beta_0 = \sum_{i=0}^{n-1}y_i - \sum_{i=0}^{n-1} X_{i1} \beta_1.
 \]
 !et
-Assuming that the matrix elements $X_{i1}$ are centered, what we have is
+We obtain then
 !bt
 \[
-\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} \left(X_{i1}-\mu_{1}\right),
+\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} X_{i1}.
 \]
 !et
-where
+If we define
 !bt
 \[
 \mu_1=\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1},
@@ -1323,25 +1310,19 @@ and if we define the mean value of the outputs as
 we have
 !bt
 \[
-\beta_0 = \mu_y - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1}-\mu_{1}),
+\beta_0 = \mu_y - \beta_1\mu_{1}.
 \]
 !et
-and it is easy to see that the last sum equals zero!  This means that we have
+In the general case withmore parameters than $\beta_0$ and $\beta_1$, we have
 !bt
 \[
-\beta_0 = \mu_y,
+\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \frac{1}{n}\sum_{i=0}^{n-1}\sum_{j=1}^{p-1} X_{ij}\beta_j.
 \]
 !et
-if the columns of the design matrix are centered. It is straight forward to generalize this results to more values of $\beta$.
-We have thus
-!bt
-\[
-\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1} y_i = \overline{\bm{y}},
-\]
-!et
-the average value of $\bm{y}$.
 
-Replacing $y_i$ with $y_i - \beta_0 = y_i - \overline{\bm{y}}$ and centering also our design matrix results in a cost function (in vector-matrix disguise)
+
+
+Replacing $y_i$ with $y_i - y_i - \overline{\bm{y}}$ and centering also our design matrix results in a cost function (in vector-matrix disguise)
 !bt
 \[
 C(\boldsymbol{\beta}) = (\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta})^T(\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta}). 
@@ -1349,6 +1330,7 @@ C(\boldsymbol{\beta}) = (\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta})^T
 !et
 
 
+
 If we minimize with respect to $\bm{\beta}$ we have then
 
 !bt
@@ -1629,8 +1611,6 @@ for i in range(nlambdas):
     OwnRidgeBeta = np.linalg.pinv(X_train_scaled.T @ X_train_scaled+lmb*I) @ X_train_scaled.T @ (y_train_scaled)
     intercept_ = y_scaler - X_train_mean@OwnRidgeBeta #The intercept can be shifted so the model can predict on uncentered data
     #Add intercept to prediction
-    ypredictOwnRidge = X_test @ OwnRidgeBeta + intercept_ 
-    #Add intercept to prediction
     ypredictOwnRidge = X_test_scaled @ OwnRidgeBeta + y_scaler 
     RegRidge = linear_model.Ridge(lmb)
     RegRidge.fit(X_train,y_train)
@@ -2463,3 +2443,4 @@ of data presented here (either the terrain data we propose or other data sets).
 
 
 
+

doc/LectureNotes/_build/jupyter_execute/Clustering.ipynb

@@ -475,7 +475,7 @@
      "output_type": "stream",
      "text": [
       "Converged at iteration 5\n",
-      "Runtime: 0.4849228858947754 seconds\n"
+      "Runtime: 0.47818493843078613 seconds\n"
      ]
     }
    ],
@@ -604,7 +604,7 @@
      "output_type": "stream",
      "text": [
       "Converged at iteration: 5\n",
-      "Runtime: 0.41676807403564453 seconds\n"
+      "Runtime: 0.42156386375427246 seconds\n"
      ]
     }
    ],
@@ -745,7 +745,7 @@
      "output_type": "stream",
      "text": [
       "Converged at iteration: 11\n",
-      "Runtime: 0.8447701930999756 seconds\n",
+      "Runtime: 0.8341619968414307 seconds\n",
       " "
      ]
     }
@@ -877,7 +877,7 @@
      "output_type": "stream",
      "text": [
       "Converged at iteration: 5\n",
-      "Runtime: 0.004042863845825195 seconds\n"
+      "Runtime: 0.0038292407989501953 seconds\n"
      ]
     }
    ],

doc/LectureNotes/_build/jupyter_execute/chapter10.ipynb

@@ -2284,15 +2284,15 @@
       "Learning rate  =  0.1\n",
       "Lambda =  1e-05\n",
       "Accuracy score on test set:  0.9111111111111111\n",
-      "\n"
+      "\n",
+      "Learning rate  =  0.1\n",
+      "Lambda =  0.0001\n"
      ]
     },
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Learning rate  =  0.1\n",
-      "Lambda =  0.0001\n",
       "Accuracy score on test set:  0.9222222222222223\n",
       "\n",
       "Learning rate  =  0.1\n",

doc/LectureNotes/chapter3.ipynb

@@ -1708,42 +1708,6 @@
     "$$"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "We assume \n",
-    "that every column of $\\boldsymbol{X}$ is centered, which we can do by subtracting the mean,"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "metadata": {
-    "collapsed": false,
-    "editable": true
-   },
-   "outputs": [],
-   "source": [
-    "X = X - np.mean(X,axis=0)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "This means that we need to rewrite $X_{ij}$ as $\\tilde{X}_{ij}=X_{ij}-\\mu_j$, where"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "$$\n",
-    "\\mu_j = \\frac{1}{n}\\sum_{i=0}^{n-1}X_{ij}.\n",
-    "$$"
-   ]
-  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -1765,7 +1729,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Assuming that the matrix elements $X_{i1}$ are centered, what we have is"
+    "We obtain then"
    ]
   },
   {
@@ -1773,7 +1737,7 @@
    "metadata": {},
    "source": [
     "$$\n",
-    "\\beta_0 = \\frac{1}{n}\\sum_{i=0}^{n-1}y_i - \\beta_1\\frac{1}{n}\\sum_{i=0}^{n-1} \\left(X_{i1}-\\mu_{1}\\right),\n",
+    "\\beta_0 = \\frac{1}{n}\\sum_{i=0}^{n-1}y_i - \\beta_1\\frac{1}{n}\\sum_{i=0}^{n-1} X_{i1}.\n",
     "$$"
    ]
   },
@@ -1781,7 +1745,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "where"
+    "If we define"
    ]
   },
   {
@@ -1821,7 +1785,7 @@
    "metadata": {},
    "source": [
     "$$\n",
-    "\\beta_0 = \\mu_y - \\beta_1\\frac{1}{n}\\sum_{i=0}^{n-1} (X_{i1}-\\mu_{1}),\n",
+    "\\beta_0 = \\mu_y - \\beta_1\\mu_{1}.\n",
     "$$"
    ]
   },
@@ -1829,7 +1793,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "and it is easy to see that the last sum equals zero!  This means that we have"
+    "In the general case withmore parameters than $\\beta_0$ and $\\beta_1$, we have"
    ]
   },
   {
@@ -1837,7 +1801,7 @@
    "metadata": {},
    "source": [
     "$$\n",
-    "\\beta_0 = \\mu_y,\n",
+    "\\beta_0 = \\frac{1}{n}\\sum_{i=0}^{n-1}y_i - \\frac{1}{n}\\sum_{i=0}^{n-1}\\sum_{j=1}^{p-1} X_{ij}\\beta_j.\n",
     "$$"
    ]
   },
@@ -1845,26 +1809,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "if the columns of the design matrix are centered. It is straight forward to generalize this results to more values of $\\beta$.\n",
-    "We have thus"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "$$\n",
-    "\\beta_0 = \\frac{1}{n}\\sum_{i=0}^{n-1} y_i = \\overline{\\boldsymbol{y}},\n",
-    "$$"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": [
-    "the average value of $\\boldsymbol{y}$.\n",
-    "\n",
-    "Replacing $y_i$ with $y_i - \\beta_0 = y_i - \\overline{\\boldsymbol{y}}$ and centering also our design matrix results in a cost function (in vector-matrix disguise)"
+    "Replacing $y_i$ with $y_i - y_i - \\overline{\\boldsymbol{y}}$ and centering also our design matrix results in a cost function (in vector-matrix disguise)"
    ]
   },
   {
@@ -2238,8 +2183,6 @@
     "    OwnRidgeBeta = np.linalg.pinv(X_train_scaled.T @ X_train_scaled+lmb*I) @ X_train_scaled.T @ (y_train_scaled)\n",
     "    intercept_ = y_scaler - X_train_mean@OwnRidgeBeta #The intercept can be shifted so the model can predict on uncentered data\n",
     "    #Add intercept to prediction\n",
-    "    ypredictOwnRidge = X_test @ OwnRidgeBeta + intercept_ \n",
-    "    #Add intercept to prediction\n",
     "    ypredictOwnRidge = X_test_scaled @ OwnRidgeBeta + y_scaler \n",
     "    RegRidge = linear_model.Ridge(lmb)\n",
     "    RegRidge.fit(X_train,y_train)\n",
@@ -2925,8 +2868,8 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "6\n",
-    "0\n",
+    "5\n",
+    "8\n",
     " \n",
     "<\n",
     "<\n",