added more material to week 35

doc/src/week35/week35.do.txt

@@ -999,13 +999,14 @@ print(XPandas.mean())
 print(XPandas.std())
 XPandas = (XPandas -XPandas.mean())
 display(XPandas)
+#  This option does not include the standard deviation
 scaler = StandardScaler(with_std=False)
 scaler.fit(X)
 Xscaled = scaler.transform(X)
 display(XPandas-Xscaled)
 !ec
 
-Small exercise: perform the standard scaling by including the standard deviation.
+Small exercise: perform the standard scaling by including the standard deviation and compare with what Scikit-Learn gives.
 
 !split
 ===== Min-Max Scaling =====
@@ -1189,30 +1190,32 @@ print("R2 score for  scaled data: {:.2f}".format(clf.score(X_test_scaled,y_test)
 What is presented here is a mathematical analysis of various regression algorithms (ordinary least  squares, Ridge and Lasso Regression). The analysis is based on an important algorithm in linear algebra, the so-called Singular Value Decomposition (SVD). 
 
 
-We have shown that in ordinary least squares that the optimal parameter $\beta$ are given by
+We have shown that in ordinary least squares the optimal parameters $\beta$ are given by
 
 !bt
 \[
-\hat{\bm{\beta}} = \left(\bm{X}^T\bm{X}\right^{-1}\bm{X}^T\bm{y}.
+\hat{\bm{\beta}} = \left(\bm{X}^T\bm{X}\right)^{-1}\bm{X}^T\bm{y}.
 \]
 !et
 
+The _hat_ over $\bm{\beta}$ means we have the optimal parameters after minimization of the cost function.
+
 This means that our best model is defined as
 
 !bt
 \[
-\tilde{\bm{y}}=\bm{X}\hat{\bm{\beta}} = \bm{X}\left(\bm{X}^T\bm{X}\right^{-1}\bm{X}^T\bm{y}.
+\tilde{\bm{y}}=\bm{X}\hat{\bm{\beta}} = \bm{X}\left(\bm{X}^T\bm{X}\right)^{-1}\bm{X}^T\bm{y}.
 \]
 !et
 
-We define now a matrix
+We now define a matrix
 !bt
 \[
-\bm{A}=\bm{X}\left(\bm{X}^T\bm{X}\right^{-1}\bm{X}^T.
+\bm{A}=\bm{X}\left(\bm{X}^T\bm{X}\right)^{-1}\bm{X}^T.
 \]
 !et
 
-It means that we can rewrite
+We can rewrite
 !bt
 \[
 \tilde{\bm{y}}=\bm{X}\hat{\bm{\beta}} = \bm{A}\bm{y}.
@@ -1220,7 +1223,9 @@ It means that we can rewrite
 !et
 
 The matrix $\bm{A}$ has the important property that $\bm{A}^2=\bm{A}$. This is the definition of a projection matrix.
-We can then interpret that our optimal model $\tilde{\bm{y}}$ ir represented  by an orthogonal (it has to be a square matrix like ours, if not we have an oblique projection matrix ) projection of $\bm{y}$ onto a space defined by the column vectors of $\bm{X}$.
+We can then interpret our optimal model $\tilde{\bm{y}}$ as being represented  by an orthogonal  projection of $\bm{y}$ onto a space defined by the column vectors of $\bm{X}$.  In our case here the matrix $\bm{A}$ is a square matrix. If it is a general rectangular matrix we have an oblique projection matrix.
+
+
 
 !split
 ===== Residual Error =====
@@ -1232,7 +1237,7 @@ We have defined the residual error as
 \]
 !et
 
-The residual errors are then the projections of $\bm{y}$ onto the orthogonal components of the space defined by the column vectors of $\bm{X}$.
+The residual errors are then the projections of $\bm{y}$ onto the orthogonal component of the space defined by the column vectors of $\bm{X}$.
 
 !split
 ===== Simple case =====
@@ -1248,7 +1253,7 @@ If the matrix $\bm{X}$ is an orthogonal (or unitary in case of complex values) m
 In this case the matrix $\bm{A}$ becomes
 !bt
 \[
-\bm{A}=\bm{X}\left(\bm{X}^T\bm{X}\right^{-1}\bm{X}^T)=\bm{I},
+\bm{A}=\bm{X}\left(\bm{X}^T\bm{X}\right)^{-1}\bm{X}^T)=\bm{I},
 \]
 !et
 and we have the obvious case
@@ -1258,7 +1263,7 @@ and we have the obvious case
 \]
 !et
 
-
+This serves also as a useful test of our codes. 
 
 !split
 ===== The singular value decomposition  =====