From d65046c133cf93ee46541fec88b336b4bd5659aa Mon Sep 17 00:00:00 2001
From: Morten Hjorth-Jensen <morten.hjorth-jensen@fys.uio.no>
Date: Sun, 24 Aug 2025 22:31:49 +0200
Subject: [PATCH] update

---
 doc/pub/week35/html/._week35-bs002.html      |   7 +-
 doc/pub/week35/html/._week35-bs003.html      |   8 +-
 doc/pub/week35/html/._week35-bs004.html      |   8 +-
 doc/pub/week35/html/._week35-bs005.html      |  16 +-
 doc/pub/week35/html/._week35-bs006.html      |   6 +-
 doc/pub/week35/html/._week35-bs013.html      |  22 +-
 doc/pub/week35/html/._week35-bs015.html      |   6 +-
 doc/pub/week35/html/._week35-bs016.html      |   8 +-
 doc/pub/week35/html/._week35-bs017.html      |   4 +-
 doc/pub/week35/html/._week35-bs018.html      |   2 +-
 doc/pub/week35/html/._week35-bs031.html      |  10 +-
 doc/pub/week35/html/._week35-bs036.html      |   6 +-
 doc/pub/week35/html/._week35-bs046.html      |   2 +-
 doc/pub/week35/html/._week35-bs058.html      |  22 +-
 doc/pub/week35/html/._week35-bs059.html      |  20 +-
 doc/pub/week35/html/._week35-bs061.html      |   4 +-
 doc/pub/week35/html/._week35-bs062.html      |  12 +-
 doc/pub/week35/html/week35-reveal.html       | 163 ++--
 doc/pub/week35/html/week35-solarized.html    | 163 ++--
 doc/pub/week35/html/week35.html              | 163 ++--
 doc/pub/week35/ipynb/ipynb-week35-src.tar.gz | Bin 192 -> 192 bytes
 doc/pub/week35/ipynb/week35.ipynb            | 891 +++++++++----------
 doc/src/week35/week35.do.txt                 | 163 ++--
 23 files changed, 850 insertions(+), 856 deletions(-)

diff --git a/doc/pub/week35/html/._week35-bs002.html b/doc/pub/week35/html/._week35-bs002.html
index 50cc554bf..21440aaf6 100644
--- a/doc/pub/week35/html/._week35-bs002.html
+++ b/doc/pub/week35/html/._week35-bs002.html
@@ -323,13 +323,12 @@ MathJax.Hub.Config({
 <!-- !split -->
 <h2 id="reminder-from-last-week" class="anchor">Reminder from last week </h2>
 
-<p>We need first a reminder from last week about linear regression. </p>
+<p>We need first a reminder from last week about linear regression. We are going to fit a continuous function with linear parameterization in terms of the parameters  \( \boldsymbol{\theta} \) and our first encounter is ordinary least squares.</p>
 
-<p>Fitting a continuous function with linear parameterization in terms of the parameters  \( \boldsymbol{\beta} \).</p>
 <ul>
-<li> Method of choice for fitting a continuous function!</li>
+<li> It is the method of choice for fitting a continuous function</li>
 <li> Gives an excellent introduction to central Machine Learning features with <b>understandable pedagogical</b> links to other methods like <b>Neural Networks</b>, <b>Support Vector Machines</b> etc</li>
-<li> Analytical expression for the fitting parameters \( \boldsymbol{\beta} \)</li>
+<li> Analytical expression for the fitting parameters \( \boldsymbol{\theta} \)</li>
 <li> Analytical expressions for statistical propertiers like mean values, variances, confidence intervals and more</li>
 <li> Analytical relation with probabilistic interpretations</li> 
 <li> Easy to introduce basic concepts like bias-variance tradeoff, cross-validation, resampling and regularization techniques and many other ML topics</li>
diff --git a/doc/pub/week35/html/._week35-bs003.html b/doc/pub/week35/html/._week35-bs003.html
index 3c3677438..62bf35a4f 100644
--- a/doc/pub/week35/html/._week35-bs003.html
+++ b/doc/pub/week35/html/._week35-bs003.html
@@ -346,19 +346,19 @@ value and a variance \( \sigma^2 \).
 <p>In linear regression we approximate the unknown function with another
 continuous function \( \tilde{\boldsymbol{y}}(\boldsymbol{x}) \) which depends linearly on
 some unknown parameters
-\( \boldsymbol{\beta}^T=[\beta_0,\beta_1,\beta_2,\dots,\beta_{p-1}] \).
+\( \boldsymbol{\theta}^T=[\theta_0,\theta_1,\theta_2,\dots,\theta_{p-1}] \).
 </p>
 
 <p>Last week we introduced the so-called design matrix in order to define
 the approximation \( \boldsymbol{\tilde{y}} \) via the unknown quantity
-\( \boldsymbol{\beta} \) as
+\( \boldsymbol{\theta} \) as
 </p>
 
 $$
-\boldsymbol{\tilde{y}}= \boldsymbol{X}\boldsymbol{\beta},
+\boldsymbol{\tilde{y}}= \boldsymbol{X}\boldsymbol{\theta},
 $$
 
-<p>and in order to find the optimal parameters \( \beta_i \) we defined a function which
+<p>and in order to find the optimal parameters \( \theta_i \) we defined a function which
 gives a measure of the spread between the values \( y_i \) (which
 represent the output values we want to reproduce) and the parametrized
 values \( \tilde{y}_i \), namely the so-called cost/loss function.
diff --git a/doc/pub/week35/html/._week35-bs004.html b/doc/pub/week35/html/._week35-bs004.html
index 4bb20012a..fa87622ea 100644
--- a/doc/pub/week35/html/._week35-bs004.html
+++ b/doc/pub/week35/html/._week35-bs004.html
@@ -325,12 +325,12 @@ MathJax.Hub.Config({
 
 <p>We used the mean squared error to define the way we measure the quality of our model</p>
 $$
-C(\boldsymbol{\beta})=\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)^T\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)\right\},
+C(\boldsymbol{\theta})=\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)^T\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)\right\},
 $$
 
 <p>or using the matrix \( \boldsymbol{X} \) and in a more compact matrix-vector notation as</p>
 $$
-C(\boldsymbol{\beta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right\}.
+C(\boldsymbol{\theta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right\}.
 $$
 
 <p>This function represents one of many possible ways to define the so-called cost function.</p>
@@ -340,10 +340,10 @@ the function \( C \) as
 </p>
 
 $$
-C(\boldsymbol{\beta})=\frac{1}{2n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2,
+C(\boldsymbol{\theta})=\frac{1}{2n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2,
 $$
 
-<p>since when taking the first derivative with respect to the unknown parameters \( \beta \), the factor of \( 2 \) cancels out. </p>
+<p>since when taking the first derivative with respect to the unknown parameters \( \theta \), the factor of \( 2 \) cancels out. </p>
 
 <p>
 <!-- navigation buttons at the bottom of the page -->
diff --git a/doc/pub/week35/html/._week35-bs005.html b/doc/pub/week35/html/._week35-bs005.html
index 07ab45faf..a3517a3a3 100644
--- a/doc/pub/week35/html/._week35-bs005.html
+++ b/doc/pub/week35/html/._week35-bs005.html
@@ -325,14 +325,14 @@ MathJax.Hub.Config({
 
 <p>The function </p>
 $$
-C(\boldsymbol{\beta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right\},
+C(\boldsymbol{\theta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right\},
 $$
 
 <p>can be linked to the variance of the quantity \( y_i \) if we interpret the latter as the mean value. 
 When linking (see the discussions next week) with the maximum likelihood approach below, we will indeed interpret \( y_i \) as a mean value
 </p>
 $$
-y_{i}=\langle y_i \rangle = \beta_0x_{i,0}+\beta_1x_{i,1}+\beta_2x_{i,2}+\dots+\beta_{n-1}x_{i,n-1}+\epsilon_i,
+y_{i}=\langle y_i \rangle = \theta_0x_{i,0}+\theta_1x_{i,1}+\theta_2x_{i,2}+\dots+\theta_{n-1}x_{i,n-1}+\epsilon_i,
 $$
 
 <p>where \( \langle y_i \rangle \) is the mean value. Keep in mind also that
@@ -345,25 +345,25 @@ the standard deviation discussed earlier. In the discussion here we
 will treat \( y_i \) as our exact value for the response variable.
 </p>
 
-<p>In order to find the parameters \( \beta_i \) we will then minimize the spread of \( C(\boldsymbol{\beta}) \), that is we are going to solve the problem</p>
+<p>In order to find the parameters \( \theta_i \) we will then minimize the spread of \( C(\boldsymbol{\theta}) \), that is we are going to solve the problem</p>
 $$
-{\displaystyle \min_{\boldsymbol{\beta}\in
-{\mathbb{R}}^{p}}}\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right\}.
+{\displaystyle \min_{\boldsymbol{\theta}\in
+{\mathbb{R}}^{p}}}\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right\}.
 $$
 
 <p>In practical terms it means we will require</p>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \beta_j} = \frac{\partial }{\partial \beta_j}\left[ \frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\beta_0x_{i,0}-\beta_1x_{i,1}-\beta_2x_{i,2}-\dots-\beta_{n-1}x_{i,n-1}\right)^2\right]=0, 
+\frac{\partial C(\boldsymbol{\theta})}{\partial \theta_j} = \frac{\partial }{\partial \theta_j}\left[ \frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\theta_0x_{i,0}-\theta_1x_{i,1}-\theta_2x_{i,2}-\dots-\theta_{n-1}x_{i,n-1}\right)^2\right]=0, 
 $$
 
 <p>which results in</p>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \beta_j} = -\frac{2}{n}\left[ \sum_{i=0}^{n-1}x_{ij}\left(y_i-\beta_0x_{i,0}-\beta_1x_{i,1}-\beta_2x_{i,2}-\dots-\beta_{n-1}x_{i,n-1}\right)\right]=0, 
+\frac{\partial C(\boldsymbol{\theta})}{\partial \theta_j} = -\frac{2}{n}\left[ \sum_{i=0}^{n-1}x_{ij}\left(y_i-\theta_0x_{i,0}-\theta_1x_{i,1}-\theta_2x_{i,2}-\dots-\theta_{n-1}x_{i,n-1}\right)\right]=0, 
 $$
 
 <p>or in a matrix-vector form as (multiplying away the factor \( -2/n \), see derivation below)</p>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}^T} = 0 = \boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right).  
+\frac{\partial C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}^T} = 0 = \boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right).  
 $$
 
 
diff --git a/doc/pub/week35/html/._week35-bs006.html b/doc/pub/week35/html/._week35-bs006.html
index b7801c390..b02176fd6 100644
--- a/doc/pub/week35/html/._week35-bs006.html
+++ b/doc/pub/week35/html/._week35-bs006.html
@@ -327,17 +327,17 @@ MathJax.Hub.Config({
 <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
 <p>We can rewrite, see the derivations below, </p>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}^T} = 0 = \boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right),  
+\frac{\partial C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}^T} = 0 = \boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right),  
 $$
 
 <p>as</p>
 $$
-\boldsymbol{X}^T\boldsymbol{y} = \boldsymbol{X}^T\boldsymbol{X}\boldsymbol{\beta},  
+\boldsymbol{X}^T\boldsymbol{y} = \boldsymbol{X}^T\boldsymbol{X}\boldsymbol{\theta},  
 $$
 
 <p>and if the matrix \( \boldsymbol{X}^T\boldsymbol{X} \) is invertible we have the solution</p>
 $$
-\boldsymbol{\beta} =\left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
+\boldsymbol{\theta} =\left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
 $$
 
 <p>We note also that since our design matrix is defined as \( \boldsymbol{X}\in
diff --git a/doc/pub/week35/html/._week35-bs013.html b/doc/pub/week35/html/._week35-bs013.html
index 7023a8f1b..cbc52604b 100644
--- a/doc/pub/week35/html/._week35-bs013.html
+++ b/doc/pub/week35/html/._week35-bs013.html
@@ -325,46 +325,46 @@ MathJax.Hub.Config({
 
 <p>We defined earlier a possible cost function using the mean squared error</p>
 $$
-C(\boldsymbol{\beta})=\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)^T\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)\right\},
+C(\boldsymbol{\theta})=\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)^T\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)\right\},
 $$
 
 <p>or using the design/feature matrix \( \boldsymbol{X} \) we have the more compact matrix-vector</p>
 $$
-C(\boldsymbol{\beta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right\}.
+C(\boldsymbol{\theta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right\}.
 $$
 
-<p>We note that the design matrix \( \boldsymbol{X} \) does not depend on the unknown parameters defined by the vector \( \boldsymbol{\beta} \).
-We are now interested in minimizing the cost function with respect to the unknown parameters \( \boldsymbol{\beta} \).
+<p>We note that the design matrix \( \boldsymbol{X} \) does not depend on the unknown parameters defined by the vector \( \boldsymbol{\theta} \).
+We are now interested in minimizing the cost function with respect to the unknown parameters \( \boldsymbol{\theta} \).
 </p>
 
 <p>The mean squared error is a scalar and if we use the results from example three above, we can define a new vector</p>
 $$
-\boldsymbol{w}=\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta},
+\boldsymbol{w}=\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta},
 $$
 
-<p>which depends on \( \boldsymbol{\beta} \). We rewrite the cost function as</p>
+<p>which depends on \( \boldsymbol{\theta} \). We rewrite the cost function as</p>
 $$
-C(\boldsymbol{\beta})=\frac{1}{n}\boldsymbol{w}^T\boldsymbol{w},
+C(\boldsymbol{\theta})=\frac{1}{n}\boldsymbol{w}^T\boldsymbol{w},
 $$
 
 <p>with partial derivative</p>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}}=\frac{2}{n}\boldsymbol{w}^T\frac{\partial \boldsymbol{w}}{\partial \boldsymbol{\beta}},
+\frac{\partial C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}}=\frac{2}{n}\boldsymbol{w}^T\frac{\partial \boldsymbol{w}}{\partial \boldsymbol{\theta}},
 $$
 
 <p>and using that</p>
 $$
-\frac{\partial \boldsymbol{w}}{\partial \boldsymbol{\beta}}=-\boldsymbol{X},
+\frac{\partial \boldsymbol{w}}{\partial \boldsymbol{\theta}}=-\boldsymbol{X},
 $$
 
 <p>where we used the result from example two above. Inserting the last expression we obtain</p>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}}=-\frac{2}{n}\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\boldsymbol{X},
+\frac{\partial C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}}=-\frac{2}{n}\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\boldsymbol{X},
 $$
 
 <p>or as</p>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}^T}=-\frac{2}{n}\boldsymbol{X}^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right).
+\frac{\partial C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}^T}=-\frac{2}{n}\boldsymbol{X}^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right).
 $$
 
 
diff --git a/doc/pub/week35/html/._week35-bs015.html b/doc/pub/week35/html/._week35-bs015.html
index 51c6cc4a0..f51eab71b 100644
--- a/doc/pub/week35/html/._week35-bs015.html
+++ b/doc/pub/week35/html/._week35-bs015.html
@@ -325,13 +325,13 @@ MathJax.Hub.Config({
 
 <p>A very important matrix we will meet again and again in machine
 learning is the Hessian.  It is given by the second derivative of the
-cost function with respect to the parameters \( \boldsymbol{\beta} \). Using the above
+cost function with respect to the parameters \( \boldsymbol{\theta} \). Using the above
 expression for derivatives of vectors and matrices, we find that the
 second derivative of the mean squared error as cost function is,
 </p>
 
 $$
-\frac{\partial}{\partial \boldsymbol{\beta}}\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}^T} =\frac{\partial}{\partial \boldsymbol{\beta}}\left[-\frac{2}{n}\boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right]=\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}.  
+\frac{\partial}{\partial \boldsymbol{\theta}}\frac{\partial C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}^T} =\frac{\partial}{\partial \boldsymbol{\theta}}\left[-\frac{2}{n}\boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right]=\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}.  
 $$
 
 <p>The Hessian matrix plays an important role and is defined here as</p>
@@ -342,7 +342,7 @@ $$
 
 <p>For ordinary least squares, it is inversely proportional (derivation
 next week) with the variance of the optimal parameters
-\( \hat{\boldsymbol{\beta}} \). Furthermore, we will see later this week that it is
+\( \hat{\boldsymbol{\theta}} \). Furthermore, we will see later this week that it is
 (aside the factor \( 1/n \)) equal to the covariance matrix. It plays also a very
 important role in optmization algorithms and Principal Component
 Analysis as a way to reduce the dimensionality of a machine learning/data analysis
diff --git a/doc/pub/week35/html/._week35-bs016.html b/doc/pub/week35/html/._week35-bs016.html
index 8404e4964..e7a6b33d0 100644
--- a/doc/pub/week35/html/._week35-bs016.html
+++ b/doc/pub/week35/html/._week35-bs016.html
@@ -328,20 +328,20 @@ MathJax.Hub.Config({
 <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
 <p>The residuals \( \boldsymbol{\epsilon} \) are in turn given by</p>
 $$
-\boldsymbol{\epsilon} = \boldsymbol{y}-\boldsymbol{\tilde{y}} = \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta},
+\boldsymbol{\epsilon} = \boldsymbol{y}-\boldsymbol{\tilde{y}} = \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta},
 $$
 
 <p>and with </p>
 $$
-\boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)= 0, 
+\boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)= 0, 
 $$
 
 <p>we have</p>
 $$
-\boldsymbol{X}^T\boldsymbol{\epsilon}=\boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)= 0, 
+\boldsymbol{X}^T\boldsymbol{\epsilon}=\boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)= 0, 
 $$
 
-<p>meaning that the solution for \( \boldsymbol{\beta} \) is the one which minimizes the residuals.  </p>
+<p>meaning that the solution for \( \boldsymbol{\theta} \) is the one which minimizes the residuals.  </p>
 </div>
 </div>
 
diff --git a/doc/pub/week35/html/._week35-bs017.html b/doc/pub/week35/html/._week35-bs017.html
index dc7390997..3e8bbca7f 100644
--- a/doc/pub/week35/html/._week35-bs017.html
+++ b/doc/pub/week35/html/._week35-bs017.html
@@ -328,10 +328,10 @@ we condiser a simple polynomial fit.
 We assume our data can represented by a fourth-order polynomial. For the $i$th component we have 
 </p>
 $$
-\tilde{y}_i = \beta_0+\beta_1x_i+\beta_2x_i^2+\beta_3x_i^3+\beta_4x_i^4.
+\tilde{y}_i = \theta_0+\theta_1x_i+\theta_2x_i^2+\theta_3x_i^3+\theta_4x_i^4.
 $$
 
-<p>we have five predictors/features. The first is the intercept \( \beta_0 \). The other terms are \( \beta_i \) with \( i=1,2,3,4 \). Furthermore we have \( n \) entries for each predictor. It means that our design matrix is an 
+<p>we have five predictors/features. The first is the intercept \( \theta_0 \). The other terms are \( \theta_i \) with \( i=1,2,3,4 \). Furthermore we have \( n \) entries for each predictor. It means that our design matrix is an 
 \( n\times p \) matrix \( \boldsymbol{X} \).
 </p>
 
diff --git a/doc/pub/week35/html/._week35-bs018.html b/doc/pub/week35/html/._week35-bs018.html
index 628fe5f2a..2a1386fe4 100644
--- a/doc/pub/week35/html/._week35-bs018.html
+++ b/doc/pub/week35/html/._week35-bs018.html
@@ -323,7 +323,7 @@ MathJax.Hub.Config({
 <!-- !split -->
 <h2 id="own-code-for-ordinary-least-squares" class="anchor">Own code for Ordinary Least Squares </h2>
 
-<p>It is rather straightforward to implement the matrix inversion and obtain the parameters \( \boldsymbol{\beta} \). After having defined the matrix \( \boldsymbol{X} \) and the outputs \( \boldsymbol{y} \) we have </p>
+<p>It is rather straightforward to implement the matrix inversion and obtain the parameters \( \boldsymbol{\theta} \). After having defined the matrix \( \boldsymbol{X} \) and the outputs \( \boldsymbol{y} \) we have </p>
 
 <!-- code=python (!bc pycod) typeset with pygments style "default" -->
 <div class="cell border-box-sizing code_cell rendered">
diff --git a/doc/pub/week35/html/._week35-bs031.html b/doc/pub/week35/html/._week35-bs031.html
index e5aadeb88..faa72a631 100644
--- a/doc/pub/week35/html/._week35-bs031.html
+++ b/doc/pub/week35/html/._week35-bs031.html
@@ -325,18 +325,18 @@ MathJax.Hub.Config({
 
 <p>What is presented here is a mathematical analysis of various regression algorithms (ordinary least  squares, Ridge and Lasso Regression). The analysis is based on an important algorithm in linear algebra, the so-called Singular Value Decomposition (SVD). </p>
 
-<p>We have shown that in ordinary least squares the optimal parameters \( \beta \) are given by</p>
+<p>We have shown that in ordinary least squares the optimal parameters \( \theta \) are given by</p>
 
 $$
-\hat{\boldsymbol{\beta}} = \left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
+\hat{\boldsymbol{\theta}} = \left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
 $$
 
-<p>The <b>hat</b> over \( \boldsymbol{\beta} \) means we have the optimal parameters after minimization of the cost function.</p>
+<p>The <b>hat</b> over \( \boldsymbol{\theta} \) means we have the optimal parameters after minimization of the cost function.</p>
 
 <p>This means that our best model is defined as</p>
 
 $$
-\tilde{\boldsymbol{y}}=\boldsymbol{X}\hat{\boldsymbol{\beta}} = \boldsymbol{X}\left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
+\tilde{\boldsymbol{y}}=\boldsymbol{X}\hat{\boldsymbol{\theta}} = \boldsymbol{X}\left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
 $$
 
 <p>We now define a matrix</p>
@@ -346,7 +346,7 @@ $$
 
 <p>We can rewrite</p>
 $$
-\tilde{\boldsymbol{y}}=\boldsymbol{X}\hat{\boldsymbol{\beta}} = \boldsymbol{A}\boldsymbol{y}.
+\tilde{\boldsymbol{y}}=\boldsymbol{X}\hat{\boldsymbol{\theta}} = \boldsymbol{A}\boldsymbol{y}.
 $$
 
 <p>The matrix \( \boldsymbol{A} \) has the important property that \( \boldsymbol{A}^2=\boldsymbol{A} \). This is the definition of a projection matrix.
diff --git a/doc/pub/week35/html/._week35-bs036.html b/doc/pub/week35/html/._week35-bs036.html
index 5f7291dff..f25cc3ac2 100644
--- a/doc/pub/week35/html/._week35-bs036.html
+++ b/doc/pub/week35/html/._week35-bs036.html
@@ -326,16 +326,16 @@ MathJax.Hub.Config({
 <p>If our design matrix \( \boldsymbol{X} \) which enters the linear regression problem</p>
 $$
 \begin{align}
-\boldsymbol{\beta} & =  (\boldsymbol{X}^{T} \boldsymbol{X})^{-1} \boldsymbol{X}^{T} \boldsymbol{y},
+\boldsymbol{\theta} & =  (\boldsymbol{X}^{T} \boldsymbol{X})^{-1} \boldsymbol{X}^{T} \boldsymbol{y},
 \tag{1}
 \end{align}
 $$
 
 <p>has linearly dependent column vectors, we will not be able to compute the inverse
-of \( \boldsymbol{X}^T\boldsymbol{X} \) and we cannot find the parameters (estimators) \( \beta_i \). 
+of \( \boldsymbol{X}^T\boldsymbol{X} \) and we cannot find the parameters (estimators) \( \theta_i \). 
 The estimators are only well-defined if \( (\boldsymbol{X}^{T}\boldsymbol{X})^{-1} \) exits. 
 This is more likely to happen when the matrix \( \boldsymbol{X} \) is high-dimensional. In this case it is likely to encounter a situation where 
-the regression parameters \( \beta_i \) cannot be estimated.
+the regression parameters \( \theta_i \) cannot be estimated.
 </p>
 
 <p>A cheap  <em>ad hoc</em> approach is  simply to add a small diagonal component to the matrix to invert, that is we change</p>
diff --git a/doc/pub/week35/html/._week35-bs046.html b/doc/pub/week35/html/._week35-bs046.html
index eb4ff0c8d..342cb0303 100644
--- a/doc/pub/week35/html/._week35-bs046.html
+++ b/doc/pub/week35/html/._week35-bs046.html
@@ -331,7 +331,7 @@ function, that is we have
 </p>
 
 $$
-\frac{\partial^2 C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}\partial \boldsymbol{\beta}^T} =\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}.  
+\frac{\partial^2 C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}\partial \boldsymbol{\theta}^T} =\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}.  
 $$
 
 <p>This quantity defines was what is called the Hessian matrix (the second derivative of a function we want to optimize).</p>
diff --git a/doc/pub/week35/html/._week35-bs058.html b/doc/pub/week35/html/._week35-bs058.html
index 1db5e819c..a7342d68f 100644
--- a/doc/pub/week35/html/._week35-bs058.html
+++ b/doc/pub/week35/html/._week35-bs058.html
@@ -327,13 +327,13 @@ MathJax.Hub.Config({
 our optimization problem is
 </p>
 $$
-{\displaystyle \min_{\boldsymbol{\beta}\in {\mathbb{R}}^{p}}}\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right\}.
+{\displaystyle \min_{\boldsymbol{\theta}\in {\mathbb{R}}^{p}}}\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right\}.
 $$
 
 <p>or we can state it as</p>
 $$
-{\displaystyle \min_{\boldsymbol{\beta}\in
-{\mathbb{R}}^{p}}}\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\vert\vert_2^2,
+{\displaystyle \min_{\boldsymbol{\theta}\in
+{\mathbb{R}}^{p}}}\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\vert\vert_2^2,
 $$
 
 <p>where we have used the definition of  a norm-2 vector, that is</p>
@@ -342,29 +342,29 @@ $$
 $$
 
 <p>By minimizing the above equation with respect to the parameters
-\( \boldsymbol{\beta} \) we could then obtain an analytical expression for the
-parameters \( \boldsymbol{\beta} \).  We can add a regularization parameter \( \lambda \) by
+\( \boldsymbol{\theta} \) we could then obtain an analytical expression for the
+parameters \( \boldsymbol{\theta} \).  We can add a regularization parameter \( \lambda \) by
 defining a new cost function to be optimized, that is
 </p>
 
 $$
-{\displaystyle \min_{\boldsymbol{\beta}\in
-{\mathbb{R}}^{p}}}\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\beta}\vert\vert_2^2
+{\displaystyle \min_{\boldsymbol{\theta}\in
+{\mathbb{R}}^{p}}}\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\theta}\vert\vert_2^2
 $$
 
 <p>which leads to the Ridge regression minimization problem where we
-require that \( \vert\vert \boldsymbol{\beta}\vert\vert_2^2\le t \), where \( t \) is
+require that \( \vert\vert \boldsymbol{\theta}\vert\vert_2^2\le t \), where \( t \) is
 a finite number larger than zero. By defining
 </p>
 
 $$
-C(\boldsymbol{X},\boldsymbol{\beta})=\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\beta}\vert\vert_1,
+C(\boldsymbol{X},\boldsymbol{\theta})=\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\theta}\vert\vert_1,
 $$
 
 <p>we have a new optimization equation</p>
 $$
-{\displaystyle \min_{\boldsymbol{\beta}\in
-{\mathbb{R}}^{p}}}\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\beta}\vert\vert_1
+{\displaystyle \min_{\boldsymbol{\theta}\in
+{\mathbb{R}}^{p}}}\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\theta}\vert\vert_1
 $$
 
 <p>which leads to Lasso regression. Lasso stands for least absolute shrinkage and selection operator. </p>
diff --git a/doc/pub/week35/html/._week35-bs059.html b/doc/pub/week35/html/._week35-bs059.html
index 02cdd228c..3b2f2f61f 100644
--- a/doc/pub/week35/html/._week35-bs059.html
+++ b/doc/pub/week35/html/._week35-bs059.html
@@ -326,37 +326,37 @@ MathJax.Hub.Config({
 <p>Using the matrix-vector expression for Ridge regression and dropping the parameter \( 1/n \) in front of the standard means squared error equation, we have</p>
 
 $$
-C(\boldsymbol{X},\boldsymbol{\beta})=\left\{(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})^T(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})\right\}+\lambda\boldsymbol{\beta}^T\boldsymbol{\beta},
+C(\boldsymbol{X},\boldsymbol{\theta})=\left\{(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta})^T(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta})\right\}+\lambda\boldsymbol{\theta}^T\boldsymbol{\theta},
 $$
 
 <p>and 
-taking the derivatives with respect to \( \boldsymbol{\beta} \) we obtain then
+taking the derivatives with respect to \( \boldsymbol{\theta} \) we obtain then
 a slightly modified matrix inversion problem which for finite values
 of \( \lambda \) does not suffer from singularity problems. We obtain
 the optimal parameters
 </p>
 $$
-\hat{\boldsymbol{\beta}}_{\mathrm{Ridge}} = \left(\boldsymbol{X}^T\boldsymbol{X}+\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y},
+\hat{\boldsymbol{\theta}}_{\mathrm{Ridge}} = \left(\boldsymbol{X}^T\boldsymbol{X}+\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y},
 $$
 
 <p>with \( \boldsymbol{I} \) being a \( p\times p \) identity matrix with the constraint that</p>
 
 $$
-\sum_{i=0}^{p-1} \beta_i^2 \leq t,
+\sum_{i=0}^{p-1} \theta_i^2 \leq t,
 $$
 
 <p>with \( t \) a finite positive number. </p>
 
-<p>If we keep the \( 1/n \) factor, the equation for the optimal \( \beta \) changes to</p>
+<p>If we keep the \( 1/n \) factor, the equation for the optimal \( \theta \) changes to</p>
 $$
-\hat{\boldsymbol{\beta}}_{\mathrm{Ridge}} = \left(\boldsymbol{X}^T\boldsymbol{X}+n\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
+\hat{\boldsymbol{\theta}}_{\mathrm{Ridge}} = \left(\boldsymbol{X}^T\boldsymbol{X}+n\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
 $$
 
 <p>In many textbooks the \( 1/n \) term is often omitted. Note that a library like <b>Scikit-Learn</b> does not include the \( 1/n \) factor in the setup of the cost function.</p>
 
 <p>When we compare this with the ordinary least squares result we have</p>
 $$
-\hat{\boldsymbol{\beta}}_{\mathrm{OLS}} = \left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y},
+\hat{\boldsymbol{\theta}}_{\mathrm{OLS}} = \left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y},
 $$
 
 <p>which can lead to singular matrices. However, with the SVD, we can always compute the inverse of the matrix \( \boldsymbol{X}^T\boldsymbol{X} \).</p>
@@ -365,20 +365,20 @@ $$
 modified diagonal term added to \( \boldsymbol{X}^T\boldsymbol{X} \). The consequences, in
 particular for our discussion of the bias-variance tradeoff are rather
 interesting. We will see that for specific values of \( \lambda \), we may
-even reduce the variance of the optimal parameters \( \boldsymbol{\beta} \). These topics and other related ones, will be discussed after the more linear algebra oriented analysis here.
+even reduce the variance of the optimal parameters \( \boldsymbol{\theta} \). These topics and other related ones, will be discussed after the more linear algebra oriented analysis here.
 </p>
 
 <p>Using our insights about the SVD of the design matrix \( \boldsymbol{X} \) 
 We have already analyzed the OLS solutions in terms of the eigenvectors (the columns) of the right singular value matrix \( \boldsymbol{U} \) as
 </p>
 $$
-\tilde{\boldsymbol{y}}_{\mathrm{OLS}}=\boldsymbol{X}\boldsymbol{\beta}  =\boldsymbol{U}\boldsymbol{U}^T\boldsymbol{y}.
+\tilde{\boldsymbol{y}}_{\mathrm{OLS}}=\boldsymbol{X}\boldsymbol{\theta}  =\boldsymbol{U}\boldsymbol{U}^T\boldsymbol{y}.
 $$
 
 <p>For Ridge regression this becomes</p>
 
 $$
-\tilde{\boldsymbol{y}}_{\mathrm{Ridge}}=\boldsymbol{X}\boldsymbol{\beta}_{\mathrm{Ridge}} = \boldsymbol{U\Sigma V^T}\left(\boldsymbol{V}\boldsymbol{\Sigma}^2\boldsymbol{V}^T+\lambda\boldsymbol{I} \right)^{-1}(\boldsymbol{U\Sigma V^T})^T\boldsymbol{y}=\sum_{j=0}^{p-1}\boldsymbol{u}_j\boldsymbol{u}_j^T\frac{\sigma_j^2}{\sigma_j^2+\lambda}\boldsymbol{y},
+\tilde{\boldsymbol{y}}_{\mathrm{Ridge}}=\boldsymbol{X}\boldsymbol{\theta}_{\mathrm{Ridge}} = \boldsymbol{U\Sigma V^T}\left(\boldsymbol{V}\boldsymbol{\Sigma}^2\boldsymbol{V}^T+\lambda\boldsymbol{I} \right)^{-1}(\boldsymbol{U\Sigma V^T})^T\boldsymbol{y}=\sum_{j=0}^{p-1}\boldsymbol{u}_j\boldsymbol{u}_j^T\frac{\sigma_j^2}{\sigma_j^2+\lambda}\boldsymbol{y},
 $$
 
 <p>with the vectors \( \boldsymbol{u}_j \) being the columns of \( \boldsymbol{U} \) from the SVD of the matrix \( \boldsymbol{X} \). </p>
diff --git a/doc/pub/week35/html/._week35-bs061.html b/doc/pub/week35/html/._week35-bs061.html
index d1fb26f51..f0e6527b8 100644
--- a/doc/pub/week35/html/._week35-bs061.html
+++ b/doc/pub/week35/html/._week35-bs061.html
@@ -331,13 +331,13 @@ $$
 
 <p>In this case the standard OLS results in </p>
 $$
-\boldsymbol{\beta}^{\mathrm{OLS}} = \boldsymbol{X}^T\boldsymbol{y}=\sum_{i=0}^{n-1}\boldsymbol{u}_i\boldsymbol{u}_i^T\boldsymbol{y},
+\boldsymbol{\theta}^{\mathrm{OLS}} = \boldsymbol{X}^T\boldsymbol{y}=\sum_{i=0}^{n-1}\boldsymbol{u}_i\boldsymbol{u}_i^T\boldsymbol{y},
 $$
 
 <p>and</p>
 
 $$
-\boldsymbol{\beta}^{\mathrm{Ridge}} = \left(\boldsymbol{I}+\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}=\left(1+\lambda\right)^{-1}\boldsymbol{\beta}^{\mathrm{OLS}},
+\boldsymbol{\theta}^{\mathrm{Ridge}} = \left(\boldsymbol{I}+\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}=\left(1+\lambda\right)^{-1}\boldsymbol{\theta}^{\mathrm{OLS}},
 $$
 
 <p>that is the Ridge estimator scales the OLS estimator by the inverse of a factor \( 1+\lambda \), and
diff --git a/doc/pub/week35/html/._week35-bs062.html b/doc/pub/week35/html/._week35-bs062.html
index 318c1563e..f1b1e7f4b 100644
--- a/doc/pub/week35/html/._week35-bs062.html
+++ b/doc/pub/week35/html/._week35-bs062.html
@@ -326,28 +326,28 @@ MathJax.Hub.Config({
 <p>Using the matrix-vector expression for Lasso regression, we have the following <b>cost</b> function</p>
 
 $$
-C(\boldsymbol{X},\boldsymbol{\beta})=\frac{1}{n}\left\{(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})^T(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})\right\}+\lambda\vert\vert\boldsymbol{\beta}\vert\vert_1,
+C(\boldsymbol{X},\boldsymbol{\theta})=\frac{1}{n}\left\{(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta})^T(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta})\right\}+\lambda\vert\vert\boldsymbol{\theta}\vert\vert_1,
 $$
 
-<p>Taking the derivative with respect to \( \boldsymbol{\beta} \) and recalling that the derivative of the absolute value is (we drop the boldfaced vector symbol for simplicty)</p>
+<p>Taking the derivative with respect to \( \boldsymbol{\theta} \) and recalling that the derivative of the absolute value is (we drop the boldfaced vector symbol for simplicty)</p>
 $$
-\frac{d \vert \beta\vert}{d \beta}=\mathrm{sgn}(\beta)=\left\{\begin{array}{cc} 1 & \beta > 0 \\-1 & \beta < 0, \end{array}\right.
+\frac{d \vert \theta\vert}{d \theta}=\mathrm{sgn}(\theta)=\left\{\begin{array}{cc} 1 & \theta > 0 \\-1 & \theta < 0, \end{array}\right.
 $$
 
 <p>we have that the derivative of the cost function is</p>
 
 $$
-\frac{\partial C(\boldsymbol{X},\boldsymbol{\beta})}{\partial \boldsymbol{\beta}}=-\frac{2}{n}\boldsymbol{X}^T(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})+\lambda sgn(\boldsymbol{\beta})=0,
+\frac{\partial C(\boldsymbol{X},\boldsymbol{\theta})}{\partial \boldsymbol{\theta}}=-\frac{2}{n}\boldsymbol{X}^T(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta})+\lambda sgn(\boldsymbol{\theta})=0,
 $$
 
 <p>and reordering we have</p>
 $$
-\boldsymbol{X}^T\boldsymbol{X}\boldsymbol{\beta}+\frac{n}{2}\lambda sgn(\boldsymbol{\beta})=2\boldsymbol{X}^T\boldsymbol{y}.
+\boldsymbol{X}^T\boldsymbol{X}\boldsymbol{\theta}+\frac{n}{2}\lambda sgn(\boldsymbol{\theta})=2\boldsymbol{X}^T\boldsymbol{y}.
 $$
 
 <p>We can redefine \( \lambda \) to absorb the constant \( n/2 \) and we rewrite the last equation as</p>
 $$
-\boldsymbol{X}^T\boldsymbol{X}\boldsymbol{\beta}+\lambda sgn(\boldsymbol{\beta})=2\boldsymbol{X}^T\boldsymbol{y}.
+\boldsymbol{X}^T\boldsymbol{X}\boldsymbol{\theta}+\lambda sgn(\boldsymbol{\theta})=2\boldsymbol{X}^T\boldsymbol{y}.
 $$
 
 <p>This equation does not lead to a nice analytical equation as in either Ridge regression or ordinary least squares. This equation can however be solved by using standard convex optimization algorithms using for example the Python package <a href="https://cvxopt.org/" target="_self">CVXOPT</a>. We will discuss this later. </p>
diff --git a/doc/pub/week35/html/week35-reveal.html b/doc/pub/week35/html/week35-reveal.html
index 0d940c371..cca79a06f 100644
--- a/doc/pub/week35/html/week35-reveal.html
+++ b/doc/pub/week35/html/week35-reveal.html
@@ -219,13 +219,12 @@ MathJax.Hub.Config({
 <section>
 <h2 id="reminder-from-last-week">Reminder from last week </h2>
 
-<p>We need first a reminder from last week about linear regression. </p>
+<p>We need first a reminder from last week about linear regression. We are going to fit a continuous function with linear parameterization in terms of the parameters  \( \boldsymbol{\theta} \) and our first encounter is ordinary least squares.</p>
 
-<p>Fitting a continuous function with linear parameterization in terms of the parameters  \( \boldsymbol{\beta} \).</p>
 <ul>
-<p><li> Method of choice for fitting a continuous function!</li>
+<p><li> It is the method of choice for fitting a continuous function</li>
 <p><li> Gives an excellent introduction to central Machine Learning features with <b>understandable pedagogical</b> links to other methods like <b>Neural Networks</b>, <b>Support Vector Machines</b> etc</li>
-<p><li> Analytical expression for the fitting parameters \( \boldsymbol{\beta} \)</li>
+<p><li> Analytical expression for the fitting parameters \( \boldsymbol{\theta} \)</li>
 <p><li> Analytical expressions for statistical propertiers like mean values, variances, confidence intervals and more</li>
 <p><li> Analytical relation with probabilistic interpretations</li> 
 <p><li> Easy to introduce basic concepts like bias-variance tradeoff, cross-validation, resampling and regularization techniques and many other ML topics</li>
@@ -269,21 +268,21 @@ value and a variance \( \sigma^2 \).
 <p>In linear regression we approximate the unknown function with another
 continuous function \( \tilde{\boldsymbol{y}}(\boldsymbol{x}) \) which depends linearly on
 some unknown parameters
-\( \boldsymbol{\beta}^T=[\beta_0,\beta_1,\beta_2,\dots,\beta_{p-1}] \).
+\( \boldsymbol{\theta}^T=[\theta_0,\theta_1,\theta_2,\dots,\theta_{p-1}] \).
 </p>
 
 <p>Last week we introduced the so-called design matrix in order to define
 the approximation \( \boldsymbol{\tilde{y}} \) via the unknown quantity
-\( \boldsymbol{\beta} \) as
+\( \boldsymbol{\theta} \) as
 </p>
 
 <p>&nbsp;<br>
 $$
-\boldsymbol{\tilde{y}}= \boldsymbol{X}\boldsymbol{\beta},
+\boldsymbol{\tilde{y}}= \boldsymbol{X}\boldsymbol{\theta},
 $$
 <p>&nbsp;<br>
 
-<p>and in order to find the optimal parameters \( \beta_i \) we defined a function which
+<p>and in order to find the optimal parameters \( \theta_i \) we defined a function which
 gives a measure of the spread between the values \( y_i \) (which
 represent the output values we want to reproduce) and the parametrized
 values \( \tilde{y}_i \), namely the so-called cost/loss function.
@@ -296,14 +295,14 @@ values \( \tilde{y}_i \), namely the so-called cost/loss function.
 <p>We used the mean squared error to define the way we measure the quality of our model</p>
 <p>&nbsp;<br>
 $$
-C(\boldsymbol{\beta})=\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)^T\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)\right\},
+C(\boldsymbol{\theta})=\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)^T\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)\right\},
 $$
 <p>&nbsp;<br>
 
 <p>or using the matrix \( \boldsymbol{X} \) and in a more compact matrix-vector notation as</p>
 <p>&nbsp;<br>
 $$
-C(\boldsymbol{\beta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right\}.
+C(\boldsymbol{\theta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right\}.
 $$
 <p>&nbsp;<br>
 
@@ -315,11 +314,11 @@ the function \( C \) as
 
 <p>&nbsp;<br>
 $$
-C(\boldsymbol{\beta})=\frac{1}{2n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2,
+C(\boldsymbol{\theta})=\frac{1}{2n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2,
 $$
 <p>&nbsp;<br>
 
-<p>since when taking the first derivative with respect to the unknown parameters \( \beta \), the factor of \( 2 \) cancels out. </p>
+<p>since when taking the first derivative with respect to the unknown parameters \( \theta \), the factor of \( 2 \) cancels out. </p>
 </section>
 
 <section>
@@ -328,7 +327,7 @@ $$
 <p>The function </p>
 <p>&nbsp;<br>
 $$
-C(\boldsymbol{\beta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right\},
+C(\boldsymbol{\theta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right\},
 $$
 <p>&nbsp;<br>
 
@@ -337,7 +336,7 @@ When linking (see the discussions next week) with the maximum likelihood approac
 </p>
 <p>&nbsp;<br>
 $$
-y_{i}=\langle y_i \rangle = \beta_0x_{i,0}+\beta_1x_{i,1}+\beta_2x_{i,2}+\dots+\beta_{n-1}x_{i,n-1}+\epsilon_i,
+y_{i}=\langle y_i \rangle = \theta_0x_{i,0}+\theta_1x_{i,1}+\theta_2x_{i,2}+\dots+\theta_{n-1}x_{i,n-1}+\epsilon_i,
 $$
 <p>&nbsp;<br>
 
@@ -351,32 +350,32 @@ the standard deviation discussed earlier. In the discussion here we
 will treat \( y_i \) as our exact value for the response variable.
 </p>
 
-<p>In order to find the parameters \( \beta_i \) we will then minimize the spread of \( C(\boldsymbol{\beta}) \), that is we are going to solve the problem</p>
+<p>In order to find the parameters \( \theta_i \) we will then minimize the spread of \( C(\boldsymbol{\theta}) \), that is we are going to solve the problem</p>
 <p>&nbsp;<br>
 $$
-{\displaystyle \min_{\boldsymbol{\beta}\in
-{\mathbb{R}}^{p}}}\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right\}.
+{\displaystyle \min_{\boldsymbol{\theta}\in
+{\mathbb{R}}^{p}}}\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right\}.
 $$
 <p>&nbsp;<br>
 
 <p>In practical terms it means we will require</p>
 <p>&nbsp;<br>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \beta_j} = \frac{\partial }{\partial \beta_j}\left[ \frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\beta_0x_{i,0}-\beta_1x_{i,1}-\beta_2x_{i,2}-\dots-\beta_{n-1}x_{i,n-1}\right)^2\right]=0, 
+\frac{\partial C(\boldsymbol{\theta})}{\partial \theta_j} = \frac{\partial }{\partial \theta_j}\left[ \frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\theta_0x_{i,0}-\theta_1x_{i,1}-\theta_2x_{i,2}-\dots-\theta_{n-1}x_{i,n-1}\right)^2\right]=0, 
 $$
 <p>&nbsp;<br>
 
 <p>which results in</p>
 <p>&nbsp;<br>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \beta_j} = -\frac{2}{n}\left[ \sum_{i=0}^{n-1}x_{ij}\left(y_i-\beta_0x_{i,0}-\beta_1x_{i,1}-\beta_2x_{i,2}-\dots-\beta_{n-1}x_{i,n-1}\right)\right]=0, 
+\frac{\partial C(\boldsymbol{\theta})}{\partial \theta_j} = -\frac{2}{n}\left[ \sum_{i=0}^{n-1}x_{ij}\left(y_i-\theta_0x_{i,0}-\theta_1x_{i,1}-\theta_2x_{i,2}-\dots-\theta_{n-1}x_{i,n-1}\right)\right]=0, 
 $$
 <p>&nbsp;<br>
 
 <p>or in a matrix-vector form as (multiplying away the factor \( -2/n \), see derivation below)</p>
 <p>&nbsp;<br>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}^T} = 0 = \boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right).  
+\frac{\partial C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}^T} = 0 = \boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right).  
 $$
 <p>&nbsp;<br>
 </section>
@@ -389,21 +388,21 @@ $$
 <p>We can rewrite, see the derivations below, </p>
 <p>&nbsp;<br>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}^T} = 0 = \boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right),  
+\frac{\partial C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}^T} = 0 = \boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right),  
 $$
 <p>&nbsp;<br>
 
 <p>as</p>
 <p>&nbsp;<br>
 $$
-\boldsymbol{X}^T\boldsymbol{y} = \boldsymbol{X}^T\boldsymbol{X}\boldsymbol{\beta},  
+\boldsymbol{X}^T\boldsymbol{y} = \boldsymbol{X}^T\boldsymbol{X}\boldsymbol{\theta},  
 $$
 <p>&nbsp;<br>
 
 <p>and if the matrix \( \boldsymbol{X}^T\boldsymbol{X} \) is invertible we have the solution</p>
 <p>&nbsp;<br>
 $$
-\boldsymbol{\beta} =\left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
+\boldsymbol{\theta} =\left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
 $$
 <p>&nbsp;<br>
 
@@ -652,60 +651,60 @@ $$
 <p>We defined earlier a possible cost function using the mean squared error</p>
 <p>&nbsp;<br>
 $$
-C(\boldsymbol{\beta})=\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)^T\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)\right\},
+C(\boldsymbol{\theta})=\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)^T\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)\right\},
 $$
 <p>&nbsp;<br>
 
 <p>or using the design/feature matrix \( \boldsymbol{X} \) we have the more compact matrix-vector</p>
 <p>&nbsp;<br>
 $$
-C(\boldsymbol{\beta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right\}.
+C(\boldsymbol{\theta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right\}.
 $$
 <p>&nbsp;<br>
 
-<p>We note that the design matrix \( \boldsymbol{X} \) does not depend on the unknown parameters defined by the vector \( \boldsymbol{\beta} \).
-We are now interested in minimizing the cost function with respect to the unknown parameters \( \boldsymbol{\beta} \).
+<p>We note that the design matrix \( \boldsymbol{X} \) does not depend on the unknown parameters defined by the vector \( \boldsymbol{\theta} \).
+We are now interested in minimizing the cost function with respect to the unknown parameters \( \boldsymbol{\theta} \).
 </p>
 
 <p>The mean squared error is a scalar and if we use the results from example three above, we can define a new vector</p>
 <p>&nbsp;<br>
 $$
-\boldsymbol{w}=\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta},
+\boldsymbol{w}=\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta},
 $$
 <p>&nbsp;<br>
 
-<p>which depends on \( \boldsymbol{\beta} \). We rewrite the cost function as</p>
+<p>which depends on \( \boldsymbol{\theta} \). We rewrite the cost function as</p>
 <p>&nbsp;<br>
 $$
-C(\boldsymbol{\beta})=\frac{1}{n}\boldsymbol{w}^T\boldsymbol{w},
+C(\boldsymbol{\theta})=\frac{1}{n}\boldsymbol{w}^T\boldsymbol{w},
 $$
 <p>&nbsp;<br>
 
 <p>with partial derivative</p>
 <p>&nbsp;<br>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}}=\frac{2}{n}\boldsymbol{w}^T\frac{\partial \boldsymbol{w}}{\partial \boldsymbol{\beta}},
+\frac{\partial C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}}=\frac{2}{n}\boldsymbol{w}^T\frac{\partial \boldsymbol{w}}{\partial \boldsymbol{\theta}},
 $$
 <p>&nbsp;<br>
 
 <p>and using that</p>
 <p>&nbsp;<br>
 $$
-\frac{\partial \boldsymbol{w}}{\partial \boldsymbol{\beta}}=-\boldsymbol{X},
+\frac{\partial \boldsymbol{w}}{\partial \boldsymbol{\theta}}=-\boldsymbol{X},
 $$
 <p>&nbsp;<br>
 
 <p>where we used the result from example two above. Inserting the last expression we obtain</p>
 <p>&nbsp;<br>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}}=-\frac{2}{n}\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\boldsymbol{X},
+\frac{\partial C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}}=-\frac{2}{n}\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\boldsymbol{X},
 $$
 <p>&nbsp;<br>
 
 <p>or as</p>
 <p>&nbsp;<br>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}^T}=-\frac{2}{n}\boldsymbol{X}^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right).
+\frac{\partial C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}^T}=-\frac{2}{n}\boldsymbol{X}^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right).
 $$
 <p>&nbsp;<br>
 </section>
@@ -744,14 +743,14 @@ $$
 
 <p>A very important matrix we will meet again and again in machine
 learning is the Hessian.  It is given by the second derivative of the
-cost function with respect to the parameters \( \boldsymbol{\beta} \). Using the above
+cost function with respect to the parameters \( \boldsymbol{\theta} \). Using the above
 expression for derivatives of vectors and matrices, we find that the
 second derivative of the mean squared error as cost function is,
 </p>
 
 <p>&nbsp;<br>
 $$
-\frac{\partial}{\partial \boldsymbol{\beta}}\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}^T} =\frac{\partial}{\partial \boldsymbol{\beta}}\left[-\frac{2}{n}\boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right]=\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}.  
+\frac{\partial}{\partial \boldsymbol{\theta}}\frac{\partial C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}^T} =\frac{\partial}{\partial \boldsymbol{\theta}}\left[-\frac{2}{n}\boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right]=\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}.  
 $$
 <p>&nbsp;<br>
 
@@ -765,7 +764,7 @@ $$
 
 <p>For ordinary least squares, it is inversely proportional (derivation
 next week) with the variance of the optimal parameters
-\( \hat{\boldsymbol{\beta}} \). Furthermore, we will see later this week that it is
+\( \hat{\boldsymbol{\theta}} \). Furthermore, we will see later this week that it is
 (aside the factor \( 1/n \)) equal to the covariance matrix. It plays also a very
 important role in optmization algorithms and Principal Component
 Analysis as a way to reduce the dimensionality of a machine learning/data analysis
@@ -784,25 +783,25 @@ v
 <p>The residuals \( \boldsymbol{\epsilon} \) are in turn given by</p>
 <p>&nbsp;<br>
 $$
-\boldsymbol{\epsilon} = \boldsymbol{y}-\boldsymbol{\tilde{y}} = \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta},
+\boldsymbol{\epsilon} = \boldsymbol{y}-\boldsymbol{\tilde{y}} = \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta},
 $$
 <p>&nbsp;<br>
 
 <p>and with </p>
 <p>&nbsp;<br>
 $$
-\boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)= 0, 
+\boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)= 0, 
 $$
 <p>&nbsp;<br>
 
 <p>we have</p>
 <p>&nbsp;<br>
 $$
-\boldsymbol{X}^T\boldsymbol{\epsilon}=\boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)= 0, 
+\boldsymbol{X}^T\boldsymbol{\epsilon}=\boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)= 0, 
 $$
 <p>&nbsp;<br>
 
-<p>meaning that the solution for \( \boldsymbol{\beta} \) is the one which minimizes the residuals.  </p>
+<p>meaning that the solution for \( \boldsymbol{\theta} \) is the one which minimizes the residuals.  </p>
 </div>
 </section>
 
@@ -815,11 +814,11 @@ We assume our data can represented by a fourth-order polynomial. For the $i$th c
 </p>
 <p>&nbsp;<br>
 $$
-\tilde{y}_i = \beta_0+\beta_1x_i+\beta_2x_i^2+\beta_3x_i^3+\beta_4x_i^4.
+\tilde{y}_i = \theta_0+\theta_1x_i+\theta_2x_i^2+\theta_3x_i^3+\theta_4x_i^4.
 $$
 <p>&nbsp;<br>
 
-<p>we have five predictors/features. The first is the intercept \( \beta_0 \). The other terms are \( \beta_i \) with \( i=1,2,3,4 \). Furthermore we have \( n \) entries for each predictor. It means that our design matrix is an 
+<p>we have five predictors/features. The first is the intercept \( \theta_0 \). The other terms are \( \theta_i \) with \( i=1,2,3,4 \). Furthermore we have \( n \) entries for each predictor. It means that our design matrix is an 
 \( n\times p \) matrix \( \boldsymbol{X} \).
 </p>
 </section>
@@ -827,7 +826,7 @@ $$
 <section>
 <h2 id="own-code-for-ordinary-least-squares">Own code for Ordinary Least Squares </h2>
 
-<p>It is rather straightforward to implement the matrix inversion and obtain the parameters \( \boldsymbol{\beta} \). After having defined the matrix \( \boldsymbol{X} \) and the outputs \( \boldsymbol{y} \) we have </p>
+<p>It is rather straightforward to implement the matrix inversion and obtain the parameters \( \boldsymbol{\theta} \). After having defined the matrix \( \boldsymbol{X} \) and the outputs \( \boldsymbol{y} \) we have </p>
 
 <!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
 <div class="cell border-box-sizing code_cell rendered">
@@ -1426,21 +1425,21 @@ plt.show()
 
 <p>What is presented here is a mathematical analysis of various regression algorithms (ordinary least  squares, Ridge and Lasso Regression). The analysis is based on an important algorithm in linear algebra, the so-called Singular Value Decomposition (SVD). </p>
 
-<p>We have shown that in ordinary least squares the optimal parameters \( \beta \) are given by</p>
+<p>We have shown that in ordinary least squares the optimal parameters \( \theta \) are given by</p>
 
 <p>&nbsp;<br>
 $$
-\hat{\boldsymbol{\beta}} = \left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
+\hat{\boldsymbol{\theta}} = \left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
 $$
 <p>&nbsp;<br>
 
-<p>The <b>hat</b> over \( \boldsymbol{\beta} \) means we have the optimal parameters after minimization of the cost function.</p>
+<p>The <b>hat</b> over \( \boldsymbol{\theta} \) means we have the optimal parameters after minimization of the cost function.</p>
 
 <p>This means that our best model is defined as</p>
 
 <p>&nbsp;<br>
 $$
-\tilde{\boldsymbol{y}}=\boldsymbol{X}\hat{\boldsymbol{\beta}} = \boldsymbol{X}\left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
+\tilde{\boldsymbol{y}}=\boldsymbol{X}\hat{\boldsymbol{\theta}} = \boldsymbol{X}\left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
 $$
 <p>&nbsp;<br>
 
@@ -1454,7 +1453,7 @@ $$
 <p>We can rewrite</p>
 <p>&nbsp;<br>
 $$
-\tilde{\boldsymbol{y}}=\boldsymbol{X}\hat{\boldsymbol{\beta}} = \boldsymbol{A}\boldsymbol{y}.
+\tilde{\boldsymbol{y}}=\boldsymbol{X}\hat{\boldsymbol{\theta}} = \boldsymbol{A}\boldsymbol{y}.
 $$
 <p>&nbsp;<br>
 
@@ -1610,17 +1609,17 @@ This is equivalent to saying that the matrix \( \boldsymbol{X} \) has at least a
 <p>&nbsp;<br>
 $$
 \begin{align}
-\boldsymbol{\beta} & =  (\boldsymbol{X}^{T} \boldsymbol{X})^{-1} \boldsymbol{X}^{T} \boldsymbol{y},
+\boldsymbol{\theta} & =  (\boldsymbol{X}^{T} \boldsymbol{X})^{-1} \boldsymbol{X}^{T} \boldsymbol{y},
 \tag{1}
 \end{align}
 $$
 <p>&nbsp;<br>
 
 <p>has linearly dependent column vectors, we will not be able to compute the inverse
-of \( \boldsymbol{X}^T\boldsymbol{X} \) and we cannot find the parameters (estimators) \( \beta_i \). 
+of \( \boldsymbol{X}^T\boldsymbol{X} \) and we cannot find the parameters (estimators) \( \theta_i \). 
 The estimators are only well-defined if \( (\boldsymbol{X}^{T}\boldsymbol{X})^{-1} \) exits. 
 This is more likely to happen when the matrix \( \boldsymbol{X} \) is high-dimensional. In this case it is likely to encounter a situation where 
-the regression parameters \( \beta_i \) cannot be estimated.
+the regression parameters \( \theta_i \) cannot be estimated.
 </p>
 
 <p>A cheap  <em>ad hoc</em> approach is  simply to add a small diagonal component to the matrix to invert, that is we change</p>
@@ -2080,7 +2079,7 @@ function, that is we have
 
 <p>&nbsp;<br>
 $$
-\frac{\partial^2 C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}\partial \boldsymbol{\beta}^T} =\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}.  
+\frac{\partial^2 C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}\partial \boldsymbol{\theta}^T} =\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}.  
 $$
 <p>&nbsp;<br>
 
@@ -2691,15 +2690,15 @@ our optimization problem is
 </p>
 <p>&nbsp;<br>
 $$
-{\displaystyle \min_{\boldsymbol{\beta}\in {\mathbb{R}}^{p}}}\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right\}.
+{\displaystyle \min_{\boldsymbol{\theta}\in {\mathbb{R}}^{p}}}\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right\}.
 $$
 <p>&nbsp;<br>
 
 <p>or we can state it as</p>
 <p>&nbsp;<br>
 $$
-{\displaystyle \min_{\boldsymbol{\beta}\in
-{\mathbb{R}}^{p}}}\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\vert\vert_2^2,
+{\displaystyle \min_{\boldsymbol{\theta}\in
+{\mathbb{R}}^{p}}}\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\vert\vert_2^2,
 $$
 <p>&nbsp;<br>
 
@@ -2711,34 +2710,34 @@ $$
 <p>&nbsp;<br>
 
 <p>By minimizing the above equation with respect to the parameters
-\( \boldsymbol{\beta} \) we could then obtain an analytical expression for the
-parameters \( \boldsymbol{\beta} \).  We can add a regularization parameter \( \lambda \) by
+\( \boldsymbol{\theta} \) we could then obtain an analytical expression for the
+parameters \( \boldsymbol{\theta} \).  We can add a regularization parameter \( \lambda \) by
 defining a new cost function to be optimized, that is
 </p>
 
 <p>&nbsp;<br>
 $$
-{\displaystyle \min_{\boldsymbol{\beta}\in
-{\mathbb{R}}^{p}}}\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\beta}\vert\vert_2^2
+{\displaystyle \min_{\boldsymbol{\theta}\in
+{\mathbb{R}}^{p}}}\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\theta}\vert\vert_2^2
 $$
 <p>&nbsp;<br>
 
 <p>which leads to the Ridge regression minimization problem where we
-require that \( \vert\vert \boldsymbol{\beta}\vert\vert_2^2\le t \), where \( t \) is
+require that \( \vert\vert \boldsymbol{\theta}\vert\vert_2^2\le t \), where \( t \) is
 a finite number larger than zero. By defining
 </p>
 
 <p>&nbsp;<br>
 $$
-C(\boldsymbol{X},\boldsymbol{\beta})=\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\beta}\vert\vert_1,
+C(\boldsymbol{X},\boldsymbol{\theta})=\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\theta}\vert\vert_1,
 $$
 <p>&nbsp;<br>
 
 <p>we have a new optimization equation</p>
 <p>&nbsp;<br>
 $$
-{\displaystyle \min_{\boldsymbol{\beta}\in
-{\mathbb{R}}^{p}}}\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\beta}\vert\vert_1
+{\displaystyle \min_{\boldsymbol{\theta}\in
+{\mathbb{R}}^{p}}}\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\theta}\vert\vert_1
 $$
 <p>&nbsp;<br>
 
@@ -2759,19 +2758,19 @@ $$
 
 <p>&nbsp;<br>
 $$
-C(\boldsymbol{X},\boldsymbol{\beta})=\left\{(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})^T(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})\right\}+\lambda\boldsymbol{\beta}^T\boldsymbol{\beta},
+C(\boldsymbol{X},\boldsymbol{\theta})=\left\{(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta})^T(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta})\right\}+\lambda\boldsymbol{\theta}^T\boldsymbol{\theta},
 $$
 <p>&nbsp;<br>
 
 <p>and 
-taking the derivatives with respect to \( \boldsymbol{\beta} \) we obtain then
+taking the derivatives with respect to \( \boldsymbol{\theta} \) we obtain then
 a slightly modified matrix inversion problem which for finite values
 of \( \lambda \) does not suffer from singularity problems. We obtain
 the optimal parameters
 </p>
 <p>&nbsp;<br>
 $$
-\hat{\boldsymbol{\beta}}_{\mathrm{Ridge}} = \left(\boldsymbol{X}^T\boldsymbol{X}+\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y},
+\hat{\boldsymbol{\theta}}_{\mathrm{Ridge}} = \left(\boldsymbol{X}^T\boldsymbol{X}+\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y},
 $$
 <p>&nbsp;<br>
 
@@ -2779,16 +2778,16 @@ $$
 
 <p>&nbsp;<br>
 $$
-\sum_{i=0}^{p-1} \beta_i^2 \leq t,
+\sum_{i=0}^{p-1} \theta_i^2 \leq t,
 $$
 <p>&nbsp;<br>
 
 <p>with \( t \) a finite positive number. </p>
 
-<p>If we keep the \( 1/n \) factor, the equation for the optimal \( \beta \) changes to</p>
+<p>If we keep the \( 1/n \) factor, the equation for the optimal \( \theta \) changes to</p>
 <p>&nbsp;<br>
 $$
-\hat{\boldsymbol{\beta}}_{\mathrm{Ridge}} = \left(\boldsymbol{X}^T\boldsymbol{X}+n\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
+\hat{\boldsymbol{\theta}}_{\mathrm{Ridge}} = \left(\boldsymbol{X}^T\boldsymbol{X}+n\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
 $$
 <p>&nbsp;<br>
 
@@ -2797,7 +2796,7 @@ $$
 <p>When we compare this with the ordinary least squares result we have</p>
 <p>&nbsp;<br>
 $$
-\hat{\boldsymbol{\beta}}_{\mathrm{OLS}} = \left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y},
+\hat{\boldsymbol{\theta}}_{\mathrm{OLS}} = \left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y},
 $$
 <p>&nbsp;<br>
 
@@ -2807,7 +2806,7 @@ $$
 modified diagonal term added to \( \boldsymbol{X}^T\boldsymbol{X} \). The consequences, in
 particular for our discussion of the bias-variance tradeoff are rather
 interesting. We will see that for specific values of \( \lambda \), we may
-even reduce the variance of the optimal parameters \( \boldsymbol{\beta} \). These topics and other related ones, will be discussed after the more linear algebra oriented analysis here.
+even reduce the variance of the optimal parameters \( \boldsymbol{\theta} \). These topics and other related ones, will be discussed after the more linear algebra oriented analysis here.
 </p>
 
 <p>Using our insights about the SVD of the design matrix \( \boldsymbol{X} \) 
@@ -2815,7 +2814,7 @@ We have already analyzed the OLS solutions in terms of the eigenvectors (the col
 </p>
 <p>&nbsp;<br>
 $$
-\tilde{\boldsymbol{y}}_{\mathrm{OLS}}=\boldsymbol{X}\boldsymbol{\beta}  =\boldsymbol{U}\boldsymbol{U}^T\boldsymbol{y}.
+\tilde{\boldsymbol{y}}_{\mathrm{OLS}}=\boldsymbol{X}\boldsymbol{\theta}  =\boldsymbol{U}\boldsymbol{U}^T\boldsymbol{y}.
 $$
 <p>&nbsp;<br>
 
@@ -2823,7 +2822,7 @@ $$
 
 <p>&nbsp;<br>
 $$
-\tilde{\boldsymbol{y}}_{\mathrm{Ridge}}=\boldsymbol{X}\boldsymbol{\beta}_{\mathrm{Ridge}} = \boldsymbol{U\Sigma V^T}\left(\boldsymbol{V}\boldsymbol{\Sigma}^2\boldsymbol{V}^T+\lambda\boldsymbol{I} \right)^{-1}(\boldsymbol{U\Sigma V^T})^T\boldsymbol{y}=\sum_{j=0}^{p-1}\boldsymbol{u}_j\boldsymbol{u}_j^T\frac{\sigma_j^2}{\sigma_j^2+\lambda}\boldsymbol{y},
+\tilde{\boldsymbol{y}}_{\mathrm{Ridge}}=\boldsymbol{X}\boldsymbol{\theta}_{\mathrm{Ridge}} = \boldsymbol{U\Sigma V^T}\left(\boldsymbol{V}\boldsymbol{\Sigma}^2\boldsymbol{V}^T+\lambda\boldsymbol{I} \right)^{-1}(\boldsymbol{U\Sigma V^T})^T\boldsymbol{y}=\sum_{j=0}^{p-1}\boldsymbol{u}_j\boldsymbol{u}_j^T\frac{\sigma_j^2}{\sigma_j^2+\lambda}\boldsymbol{y},
 $$
 <p>&nbsp;<br>
 
@@ -2865,7 +2864,7 @@ $$
 <p>In this case the standard OLS results in </p>
 <p>&nbsp;<br>
 $$
-\boldsymbol{\beta}^{\mathrm{OLS}} = \boldsymbol{X}^T\boldsymbol{y}=\sum_{i=0}^{n-1}\boldsymbol{u}_i\boldsymbol{u}_i^T\boldsymbol{y},
+\boldsymbol{\theta}^{\mathrm{OLS}} = \boldsymbol{X}^T\boldsymbol{y}=\sum_{i=0}^{n-1}\boldsymbol{u}_i\boldsymbol{u}_i^T\boldsymbol{y},
 $$
 <p>&nbsp;<br>
 
@@ -2873,7 +2872,7 @@ $$
 
 <p>&nbsp;<br>
 $$
-\boldsymbol{\beta}^{\mathrm{Ridge}} = \left(\boldsymbol{I}+\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}=\left(1+\lambda\right)^{-1}\boldsymbol{\beta}^{\mathrm{OLS}},
+\boldsymbol{\theta}^{\mathrm{Ridge}} = \left(\boldsymbol{I}+\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}=\left(1+\lambda\right)^{-1}\boldsymbol{\theta}^{\mathrm{OLS}},
 $$
 <p>&nbsp;<br>
 
@@ -2896,14 +2895,14 @@ Similarly, <a href="https://arxiv.org/abs/1803.08823" target="_blank">Mehta et a
 
 <p>&nbsp;<br>
 $$
-C(\boldsymbol{X},\boldsymbol{\beta})=\frac{1}{n}\left\{(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})^T(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})\right\}+\lambda\vert\vert\boldsymbol{\beta}\vert\vert_1,
+C(\boldsymbol{X},\boldsymbol{\theta})=\frac{1}{n}\left\{(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta})^T(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta})\right\}+\lambda\vert\vert\boldsymbol{\theta}\vert\vert_1,
 $$
 <p>&nbsp;<br>
 
-<p>Taking the derivative with respect to \( \boldsymbol{\beta} \) and recalling that the derivative of the absolute value is (we drop the boldfaced vector symbol for simplicty)</p>
+<p>Taking the derivative with respect to \( \boldsymbol{\theta} \) and recalling that the derivative of the absolute value is (we drop the boldfaced vector symbol for simplicty)</p>
 <p>&nbsp;<br>
 $$
-\frac{d \vert \beta\vert}{d \beta}=\mathrm{sgn}(\beta)=\left\{\begin{array}{cc} 1 & \beta > 0 \\-1 & \beta < 0, \end{array}\right.
+\frac{d \vert \theta\vert}{d \theta}=\mathrm{sgn}(\theta)=\left\{\begin{array}{cc} 1 & \theta > 0 \\-1 & \theta < 0, \end{array}\right.
 $$
 <p>&nbsp;<br>
 
@@ -2911,21 +2910,21 @@ $$
 
 <p>&nbsp;<br>
 $$
-\frac{\partial C(\boldsymbol{X},\boldsymbol{\beta})}{\partial \boldsymbol{\beta}}=-\frac{2}{n}\boldsymbol{X}^T(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})+\lambda sgn(\boldsymbol{\beta})=0,
+\frac{\partial C(\boldsymbol{X},\boldsymbol{\theta})}{\partial \boldsymbol{\theta}}=-\frac{2}{n}\boldsymbol{X}^T(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta})+\lambda sgn(\boldsymbol{\theta})=0,
 $$
 <p>&nbsp;<br>
 
 <p>and reordering we have</p>
 <p>&nbsp;<br>
 $$
-\boldsymbol{X}^T\boldsymbol{X}\boldsymbol{\beta}+\frac{n}{2}\lambda sgn(\boldsymbol{\beta})=2\boldsymbol{X}^T\boldsymbol{y}.
+\boldsymbol{X}^T\boldsymbol{X}\boldsymbol{\theta}+\frac{n}{2}\lambda sgn(\boldsymbol{\theta})=2\boldsymbol{X}^T\boldsymbol{y}.
 $$
 <p>&nbsp;<br>
 
 <p>We can redefine \( \lambda \) to absorb the constant \( n/2 \) and we rewrite the last equation as</p>
 <p>&nbsp;<br>
 $$
-\boldsymbol{X}^T\boldsymbol{X}\boldsymbol{\beta}+\lambda sgn(\boldsymbol{\beta})=2\boldsymbol{X}^T\boldsymbol{y}.
+\boldsymbol{X}^T\boldsymbol{X}\boldsymbol{\theta}+\lambda sgn(\boldsymbol{\theta})=2\boldsymbol{X}^T\boldsymbol{y}.
 $$
 <p>&nbsp;<br>
 
diff --git a/doc/pub/week35/html/week35-solarized.html b/doc/pub/week35/html/week35-solarized.html
index b458df946..c0b90c6f4 100644
--- a/doc/pub/week35/html/week35-solarized.html
+++ b/doc/pub/week35/html/week35-solarized.html
@@ -303,13 +303,12 @@ MathJax.Hub.Config({
 <!-- !split --><br><br><br><br><br><br><br><br><br><br>
 <h2 id="reminder-from-last-week">Reminder from last week </h2>
 
-<p>We need first a reminder from last week about linear regression. </p>
+<p>We need first a reminder from last week about linear regression. We are going to fit a continuous function with linear parameterization in terms of the parameters  \( \boldsymbol{\theta} \) and our first encounter is ordinary least squares.</p>
 
-<p>Fitting a continuous function with linear parameterization in terms of the parameters  \( \boldsymbol{\beta} \).</p>
 <ul>
-<li> Method of choice for fitting a continuous function!</li>
+<li> It is the method of choice for fitting a continuous function</li>
 <li> Gives an excellent introduction to central Machine Learning features with <b>understandable pedagogical</b> links to other methods like <b>Neural Networks</b>, <b>Support Vector Machines</b> etc</li>
-<li> Analytical expression for the fitting parameters \( \boldsymbol{\beta} \)</li>
+<li> Analytical expression for the fitting parameters \( \boldsymbol{\theta} \)</li>
 <li> Analytical expressions for statistical propertiers like mean values, variances, confidence intervals and more</li>
 <li> Analytical relation with probabilistic interpretations</li> 
 <li> Easy to introduce basic concepts like bias-variance tradeoff, cross-validation, resampling and regularization techniques and many other ML topics</li>
@@ -347,19 +346,19 @@ value and a variance \( \sigma^2 \).
 <p>In linear regression we approximate the unknown function with another
 continuous function \( \tilde{\boldsymbol{y}}(\boldsymbol{x}) \) which depends linearly on
 some unknown parameters
-\( \boldsymbol{\beta}^T=[\beta_0,\beta_1,\beta_2,\dots,\beta_{p-1}] \).
+\( \boldsymbol{\theta}^T=[\theta_0,\theta_1,\theta_2,\dots,\theta_{p-1}] \).
 </p>
 
 <p>Last week we introduced the so-called design matrix in order to define
 the approximation \( \boldsymbol{\tilde{y}} \) via the unknown quantity
-\( \boldsymbol{\beta} \) as
+\( \boldsymbol{\theta} \) as
 </p>
 
 $$
-\boldsymbol{\tilde{y}}= \boldsymbol{X}\boldsymbol{\beta},
+\boldsymbol{\tilde{y}}= \boldsymbol{X}\boldsymbol{\theta},
 $$
 
-<p>and in order to find the optimal parameters \( \beta_i \) we defined a function which
+<p>and in order to find the optimal parameters \( \theta_i \) we defined a function which
 gives a measure of the spread between the values \( y_i \) (which
 represent the output values we want to reproduce) and the parametrized
 values \( \tilde{y}_i \), namely the so-called cost/loss function.
@@ -370,12 +369,12 @@ values \( \tilde{y}_i \), namely the so-called cost/loss function.
 
 <p>We used the mean squared error to define the way we measure the quality of our model</p>
 $$
-C(\boldsymbol{\beta})=\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)^T\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)\right\},
+C(\boldsymbol{\theta})=\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)^T\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)\right\},
 $$
 
 <p>or using the matrix \( \boldsymbol{X} \) and in a more compact matrix-vector notation as</p>
 $$
-C(\boldsymbol{\beta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right\}.
+C(\boldsymbol{\theta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right\}.
 $$
 
 <p>This function represents one of many possible ways to define the so-called cost function.</p>
@@ -385,24 +384,24 @@ the function \( C \) as
 </p>
 
 $$
-C(\boldsymbol{\beta})=\frac{1}{2n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2,
+C(\boldsymbol{\theta})=\frac{1}{2n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2,
 $$
 
-<p>since when taking the first derivative with respect to the unknown parameters \( \beta \), the factor of \( 2 \) cancels out. </p>
+<p>since when taking the first derivative with respect to the unknown parameters \( \theta \), the factor of \( 2 \) cancels out. </p>
 
 <!-- !split --><br><br><br><br><br><br><br><br><br><br>
 <h2 id="interpretations-and-optimizing-our-parameters">Interpretations and optimizing our parameters  </h2>
 
 <p>The function </p>
 $$
-C(\boldsymbol{\beta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right\},
+C(\boldsymbol{\theta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right\},
 $$
 
 <p>can be linked to the variance of the quantity \( y_i \) if we interpret the latter as the mean value. 
 When linking (see the discussions next week) with the maximum likelihood approach below, we will indeed interpret \( y_i \) as a mean value
 </p>
 $$
-y_{i}=\langle y_i \rangle = \beta_0x_{i,0}+\beta_1x_{i,1}+\beta_2x_{i,2}+\dots+\beta_{n-1}x_{i,n-1}+\epsilon_i,
+y_{i}=\langle y_i \rangle = \theta_0x_{i,0}+\theta_1x_{i,1}+\theta_2x_{i,2}+\dots+\theta_{n-1}x_{i,n-1}+\epsilon_i,
 $$
 
 <p>where \( \langle y_i \rangle \) is the mean value. Keep in mind also that
@@ -415,25 +414,25 @@ the standard deviation discussed earlier. In the discussion here we
 will treat \( y_i \) as our exact value for the response variable.
 </p>
 
-<p>In order to find the parameters \( \beta_i \) we will then minimize the spread of \( C(\boldsymbol{\beta}) \), that is we are going to solve the problem</p>
+<p>In order to find the parameters \( \theta_i \) we will then minimize the spread of \( C(\boldsymbol{\theta}) \), that is we are going to solve the problem</p>
 $$
-{\displaystyle \min_{\boldsymbol{\beta}\in
-{\mathbb{R}}^{p}}}\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right\}.
+{\displaystyle \min_{\boldsymbol{\theta}\in
+{\mathbb{R}}^{p}}}\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right\}.
 $$
 
 <p>In practical terms it means we will require</p>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \beta_j} = \frac{\partial }{\partial \beta_j}\left[ \frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\beta_0x_{i,0}-\beta_1x_{i,1}-\beta_2x_{i,2}-\dots-\beta_{n-1}x_{i,n-1}\right)^2\right]=0, 
+\frac{\partial C(\boldsymbol{\theta})}{\partial \theta_j} = \frac{\partial }{\partial \theta_j}\left[ \frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\theta_0x_{i,0}-\theta_1x_{i,1}-\theta_2x_{i,2}-\dots-\theta_{n-1}x_{i,n-1}\right)^2\right]=0, 
 $$
 
 <p>which results in</p>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \beta_j} = -\frac{2}{n}\left[ \sum_{i=0}^{n-1}x_{ij}\left(y_i-\beta_0x_{i,0}-\beta_1x_{i,1}-\beta_2x_{i,2}-\dots-\beta_{n-1}x_{i,n-1}\right)\right]=0, 
+\frac{\partial C(\boldsymbol{\theta})}{\partial \theta_j} = -\frac{2}{n}\left[ \sum_{i=0}^{n-1}x_{ij}\left(y_i-\theta_0x_{i,0}-\theta_1x_{i,1}-\theta_2x_{i,2}-\dots-\theta_{n-1}x_{i,n-1}\right)\right]=0, 
 $$
 
 <p>or in a matrix-vector form as (multiplying away the factor \( -2/n \), see derivation below)</p>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}^T} = 0 = \boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right).  
+\frac{\partial C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}^T} = 0 = \boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right).  
 $$
 
 
@@ -444,17 +443,17 @@ $$
 <p>
 <p>We can rewrite, see the derivations below, </p>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}^T} = 0 = \boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right),  
+\frac{\partial C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}^T} = 0 = \boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right),  
 $$
 
 <p>as</p>
 $$
-\boldsymbol{X}^T\boldsymbol{y} = \boldsymbol{X}^T\boldsymbol{X}\boldsymbol{\beta},  
+\boldsymbol{X}^T\boldsymbol{y} = \boldsymbol{X}^T\boldsymbol{X}\boldsymbol{\theta},  
 $$
 
 <p>and if the matrix \( \boldsymbol{X}^T\boldsymbol{X} \) is invertible we have the solution</p>
 $$
-\boldsymbol{\beta} =\left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
+\boldsymbol{\theta} =\left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
 $$
 
 <p>We note also that since our design matrix is defined as \( \boldsymbol{X}\in
@@ -662,46 +661,46 @@ $$
 
 <p>We defined earlier a possible cost function using the mean squared error</p>
 $$
-C(\boldsymbol{\beta})=\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)^T\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)\right\},
+C(\boldsymbol{\theta})=\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)^T\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)\right\},
 $$
 
 <p>or using the design/feature matrix \( \boldsymbol{X} \) we have the more compact matrix-vector</p>
 $$
-C(\boldsymbol{\beta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right\}.
+C(\boldsymbol{\theta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right\}.
 $$
 
-<p>We note that the design matrix \( \boldsymbol{X} \) does not depend on the unknown parameters defined by the vector \( \boldsymbol{\beta} \).
-We are now interested in minimizing the cost function with respect to the unknown parameters \( \boldsymbol{\beta} \).
+<p>We note that the design matrix \( \boldsymbol{X} \) does not depend on the unknown parameters defined by the vector \( \boldsymbol{\theta} \).
+We are now interested in minimizing the cost function with respect to the unknown parameters \( \boldsymbol{\theta} \).
 </p>
 
 <p>The mean squared error is a scalar and if we use the results from example three above, we can define a new vector</p>
 $$
-\boldsymbol{w}=\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta},
+\boldsymbol{w}=\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta},
 $$
 
-<p>which depends on \( \boldsymbol{\beta} \). We rewrite the cost function as</p>
+<p>which depends on \( \boldsymbol{\theta} \). We rewrite the cost function as</p>
 $$
-C(\boldsymbol{\beta})=\frac{1}{n}\boldsymbol{w}^T\boldsymbol{w},
+C(\boldsymbol{\theta})=\frac{1}{n}\boldsymbol{w}^T\boldsymbol{w},
 $$
 
 <p>with partial derivative</p>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}}=\frac{2}{n}\boldsymbol{w}^T\frac{\partial \boldsymbol{w}}{\partial \boldsymbol{\beta}},
+\frac{\partial C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}}=\frac{2}{n}\boldsymbol{w}^T\frac{\partial \boldsymbol{w}}{\partial \boldsymbol{\theta}},
 $$
 
 <p>and using that</p>
 $$
-\frac{\partial \boldsymbol{w}}{\partial \boldsymbol{\beta}}=-\boldsymbol{X},
+\frac{\partial \boldsymbol{w}}{\partial \boldsymbol{\theta}}=-\boldsymbol{X},
 $$
 
 <p>where we used the result from example two above. Inserting the last expression we obtain</p>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}}=-\frac{2}{n}\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\boldsymbol{X},
+\frac{\partial C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}}=-\frac{2}{n}\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\boldsymbol{X},
 $$
 
 <p>or as</p>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}^T}=-\frac{2}{n}\boldsymbol{X}^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right).
+\frac{\partial C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}^T}=-\frac{2}{n}\boldsymbol{X}^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right).
 $$
 
 
@@ -732,13 +731,13 @@ $$
 
 <p>A very important matrix we will meet again and again in machine
 learning is the Hessian.  It is given by the second derivative of the
-cost function with respect to the parameters \( \boldsymbol{\beta} \). Using the above
+cost function with respect to the parameters \( \boldsymbol{\theta} \). Using the above
 expression for derivatives of vectors and matrices, we find that the
 second derivative of the mean squared error as cost function is,
 </p>
 
 $$
-\frac{\partial}{\partial \boldsymbol{\beta}}\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}^T} =\frac{\partial}{\partial \boldsymbol{\beta}}\left[-\frac{2}{n}\boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right]=\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}.  
+\frac{\partial}{\partial \boldsymbol{\theta}}\frac{\partial C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}^T} =\frac{\partial}{\partial \boldsymbol{\theta}}\left[-\frac{2}{n}\boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right]=\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}.  
 $$
 
 <p>The Hessian matrix plays an important role and is defined here as</p>
@@ -749,7 +748,7 @@ $$
 
 <p>For ordinary least squares, it is inversely proportional (derivation
 next week) with the variance of the optimal parameters
-\( \hat{\boldsymbol{\beta}} \). Furthermore, we will see later this week that it is
+\( \hat{\boldsymbol{\theta}} \). Furthermore, we will see later this week that it is
 (aside the factor \( 1/n \)) equal to the covariance matrix. It plays also a very
 important role in optmization algorithms and Principal Component
 Analysis as a way to reduce the dimensionality of a machine learning/data analysis
@@ -766,20 +765,20 @@ v
 <p>
 <p>The residuals \( \boldsymbol{\epsilon} \) are in turn given by</p>
 $$
-\boldsymbol{\epsilon} = \boldsymbol{y}-\boldsymbol{\tilde{y}} = \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta},
+\boldsymbol{\epsilon} = \boldsymbol{y}-\boldsymbol{\tilde{y}} = \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta},
 $$
 
 <p>and with </p>
 $$
-\boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)= 0, 
+\boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)= 0, 
 $$
 
 <p>we have</p>
 $$
-\boldsymbol{X}^T\boldsymbol{\epsilon}=\boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)= 0, 
+\boldsymbol{X}^T\boldsymbol{\epsilon}=\boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)= 0, 
 $$
 
-<p>meaning that the solution for \( \boldsymbol{\beta} \) is the one which minimizes the residuals.  </p>
+<p>meaning that the solution for \( \boldsymbol{\theta} \) is the one which minimizes the residuals.  </p>
 </div>
 
 
@@ -791,17 +790,17 @@ we condiser a simple polynomial fit.
 We assume our data can represented by a fourth-order polynomial. For the $i$th component we have 
 </p>
 $$
-\tilde{y}_i = \beta_0+\beta_1x_i+\beta_2x_i^2+\beta_3x_i^3+\beta_4x_i^4.
+\tilde{y}_i = \theta_0+\theta_1x_i+\theta_2x_i^2+\theta_3x_i^3+\theta_4x_i^4.
 $$
 
-<p>we have five predictors/features. The first is the intercept \( \beta_0 \). The other terms are \( \beta_i \) with \( i=1,2,3,4 \). Furthermore we have \( n \) entries for each predictor. It means that our design matrix is an 
+<p>we have five predictors/features. The first is the intercept \( \theta_0 \). The other terms are \( \theta_i \) with \( i=1,2,3,4 \). Furthermore we have \( n \) entries for each predictor. It means that our design matrix is an 
 \( n\times p \) matrix \( \boldsymbol{X} \).
 </p>
 
 <!-- !split --><br><br><br><br><br><br><br><br><br><br>
 <h2 id="own-code-for-ordinary-least-squares">Own code for Ordinary Least Squares </h2>
 
-<p>It is rather straightforward to implement the matrix inversion and obtain the parameters \( \boldsymbol{\beta} \). After having defined the matrix \( \boldsymbol{X} \) and the outputs \( \boldsymbol{y} \) we have </p>
+<p>It is rather straightforward to implement the matrix inversion and obtain the parameters \( \boldsymbol{\theta} \). After having defined the matrix \( \boldsymbol{X} \) and the outputs \( \boldsymbol{y} \) we have </p>
 
 <!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
 <div class="cell border-box-sizing code_cell rendered">
@@ -1391,18 +1390,18 @@ plt.show()
 
 <p>What is presented here is a mathematical analysis of various regression algorithms (ordinary least  squares, Ridge and Lasso Regression). The analysis is based on an important algorithm in linear algebra, the so-called Singular Value Decomposition (SVD). </p>
 
-<p>We have shown that in ordinary least squares the optimal parameters \( \beta \) are given by</p>
+<p>We have shown that in ordinary least squares the optimal parameters \( \theta \) are given by</p>
 
 $$
-\hat{\boldsymbol{\beta}} = \left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
+\hat{\boldsymbol{\theta}} = \left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
 $$
 
-<p>The <b>hat</b> over \( \boldsymbol{\beta} \) means we have the optimal parameters after minimization of the cost function.</p>
+<p>The <b>hat</b> over \( \boldsymbol{\theta} \) means we have the optimal parameters after minimization of the cost function.</p>
 
 <p>This means that our best model is defined as</p>
 
 $$
-\tilde{\boldsymbol{y}}=\boldsymbol{X}\hat{\boldsymbol{\beta}} = \boldsymbol{X}\left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
+\tilde{\boldsymbol{y}}=\boldsymbol{X}\hat{\boldsymbol{\theta}} = \boldsymbol{X}\left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
 $$
 
 <p>We now define a matrix</p>
@@ -1412,7 +1411,7 @@ $$
 
 <p>We can rewrite</p>
 $$
-\tilde{\boldsymbol{y}}=\boldsymbol{X}\hat{\boldsymbol{\beta}} = \boldsymbol{A}\boldsymbol{y}.
+\tilde{\boldsymbol{y}}=\boldsymbol{X}\hat{\boldsymbol{\theta}} = \boldsymbol{A}\boldsymbol{y}.
 $$
 
 <p>The matrix \( \boldsymbol{A} \) has the important property that \( \boldsymbol{A}^2=\boldsymbol{A} \). This is the definition of a projection matrix.
@@ -1550,16 +1549,16 @@ This is equivalent to saying that the matrix \( \boldsymbol{X} \) has at least a
 <p>If our design matrix \( \boldsymbol{X} \) which enters the linear regression problem</p>
 $$
 \begin{align}
-\boldsymbol{\beta} & =  (\boldsymbol{X}^{T} \boldsymbol{X})^{-1} \boldsymbol{X}^{T} \boldsymbol{y},
+\boldsymbol{\theta} & =  (\boldsymbol{X}^{T} \boldsymbol{X})^{-1} \boldsymbol{X}^{T} \boldsymbol{y},
 \label{_auto1}
 \end{align}
 $$
 
 <p>has linearly dependent column vectors, we will not be able to compute the inverse
-of \( \boldsymbol{X}^T\boldsymbol{X} \) and we cannot find the parameters (estimators) \( \beta_i \). 
+of \( \boldsymbol{X}^T\boldsymbol{X} \) and we cannot find the parameters (estimators) \( \theta_i \). 
 The estimators are only well-defined if \( (\boldsymbol{X}^{T}\boldsymbol{X})^{-1} \) exits. 
 This is more likely to happen when the matrix \( \boldsymbol{X} \) is high-dimensional. In this case it is likely to encounter a situation where 
-the regression parameters \( \beta_i \) cannot be estimated.
+the regression parameters \( \theta_i \) cannot be estimated.
 </p>
 
 <p>A cheap  <em>ad hoc</em> approach is  simply to add a small diagonal component to the matrix to invert, that is we change</p>
@@ -1956,7 +1955,7 @@ function, that is we have
 </p>
 
 $$
-\frac{\partial^2 C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}\partial \boldsymbol{\beta}^T} =\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}.  
+\frac{\partial^2 C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}\partial \boldsymbol{\theta}^T} =\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}.  
 $$
 
 <p>This quantity defines was what is called the Hessian matrix (the second derivative of a function we want to optimize).</p>
@@ -2498,13 +2497,13 @@ values and the column vectors of \( \boldsymbol{V} \).
 our optimization problem is
 </p>
 $$
-{\displaystyle \min_{\boldsymbol{\beta}\in {\mathbb{R}}^{p}}}\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right\}.
+{\displaystyle \min_{\boldsymbol{\theta}\in {\mathbb{R}}^{p}}}\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right\}.
 $$
 
 <p>or we can state it as</p>
 $$
-{\displaystyle \min_{\boldsymbol{\beta}\in
-{\mathbb{R}}^{p}}}\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\vert\vert_2^2,
+{\displaystyle \min_{\boldsymbol{\theta}\in
+{\mathbb{R}}^{p}}}\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\vert\vert_2^2,
 $$
 
 <p>where we have used the definition of  a norm-2 vector, that is</p>
@@ -2513,29 +2512,29 @@ $$
 $$
 
 <p>By minimizing the above equation with respect to the parameters
-\( \boldsymbol{\beta} \) we could then obtain an analytical expression for the
-parameters \( \boldsymbol{\beta} \).  We can add a regularization parameter \( \lambda \) by
+\( \boldsymbol{\theta} \) we could then obtain an analytical expression for the
+parameters \( \boldsymbol{\theta} \).  We can add a regularization parameter \( \lambda \) by
 defining a new cost function to be optimized, that is
 </p>
 
 $$
-{\displaystyle \min_{\boldsymbol{\beta}\in
-{\mathbb{R}}^{p}}}\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\beta}\vert\vert_2^2
+{\displaystyle \min_{\boldsymbol{\theta}\in
+{\mathbb{R}}^{p}}}\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\theta}\vert\vert_2^2
 $$
 
 <p>which leads to the Ridge regression minimization problem where we
-require that \( \vert\vert \boldsymbol{\beta}\vert\vert_2^2\le t \), where \( t \) is
+require that \( \vert\vert \boldsymbol{\theta}\vert\vert_2^2\le t \), where \( t \) is
 a finite number larger than zero. By defining
 </p>
 
 $$
-C(\boldsymbol{X},\boldsymbol{\beta})=\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\beta}\vert\vert_1,
+C(\boldsymbol{X},\boldsymbol{\theta})=\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\theta}\vert\vert_1,
 $$
 
 <p>we have a new optimization equation</p>
 $$
-{\displaystyle \min_{\boldsymbol{\beta}\in
-{\mathbb{R}}^{p}}}\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\beta}\vert\vert_1
+{\displaystyle \min_{\boldsymbol{\theta}\in
+{\mathbb{R}}^{p}}}\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\theta}\vert\vert_1
 $$
 
 <p>which leads to Lasso regression. Lasso stands for least absolute shrinkage and selection operator. </p>
@@ -2552,37 +2551,37 @@ $$
 <p>Using the matrix-vector expression for Ridge regression and dropping the parameter \( 1/n \) in front of the standard means squared error equation, we have</p>
 
 $$
-C(\boldsymbol{X},\boldsymbol{\beta})=\left\{(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})^T(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})\right\}+\lambda\boldsymbol{\beta}^T\boldsymbol{\beta},
+C(\boldsymbol{X},\boldsymbol{\theta})=\left\{(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta})^T(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta})\right\}+\lambda\boldsymbol{\theta}^T\boldsymbol{\theta},
 $$
 
 <p>and 
-taking the derivatives with respect to \( \boldsymbol{\beta} \) we obtain then
+taking the derivatives with respect to \( \boldsymbol{\theta} \) we obtain then
 a slightly modified matrix inversion problem which for finite values
 of \( \lambda \) does not suffer from singularity problems. We obtain
 the optimal parameters
 </p>
 $$
-\hat{\boldsymbol{\beta}}_{\mathrm{Ridge}} = \left(\boldsymbol{X}^T\boldsymbol{X}+\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y},
+\hat{\boldsymbol{\theta}}_{\mathrm{Ridge}} = \left(\boldsymbol{X}^T\boldsymbol{X}+\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y},
 $$
 
 <p>with \( \boldsymbol{I} \) being a \( p\times p \) identity matrix with the constraint that</p>
 
 $$
-\sum_{i=0}^{p-1} \beta_i^2 \leq t,
+\sum_{i=0}^{p-1} \theta_i^2 \leq t,
 $$
 
 <p>with \( t \) a finite positive number. </p>
 
-<p>If we keep the \( 1/n \) factor, the equation for the optimal \( \beta \) changes to</p>
+<p>If we keep the \( 1/n \) factor, the equation for the optimal \( \theta \) changes to</p>
 $$
-\hat{\boldsymbol{\beta}}_{\mathrm{Ridge}} = \left(\boldsymbol{X}^T\boldsymbol{X}+n\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
+\hat{\boldsymbol{\theta}}_{\mathrm{Ridge}} = \left(\boldsymbol{X}^T\boldsymbol{X}+n\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
 $$
 
 <p>In many textbooks the \( 1/n \) term is often omitted. Note that a library like <b>Scikit-Learn</b> does not include the \( 1/n \) factor in the setup of the cost function.</p>
 
 <p>When we compare this with the ordinary least squares result we have</p>
 $$
-\hat{\boldsymbol{\beta}}_{\mathrm{OLS}} = \left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y},
+\hat{\boldsymbol{\theta}}_{\mathrm{OLS}} = \left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y},
 $$
 
 <p>which can lead to singular matrices. However, with the SVD, we can always compute the inverse of the matrix \( \boldsymbol{X}^T\boldsymbol{X} \).</p>
@@ -2591,20 +2590,20 @@ $$
 modified diagonal term added to \( \boldsymbol{X}^T\boldsymbol{X} \). The consequences, in
 particular for our discussion of the bias-variance tradeoff are rather
 interesting. We will see that for specific values of \( \lambda \), we may
-even reduce the variance of the optimal parameters \( \boldsymbol{\beta} \). These topics and other related ones, will be discussed after the more linear algebra oriented analysis here.
+even reduce the variance of the optimal parameters \( \boldsymbol{\theta} \). These topics and other related ones, will be discussed after the more linear algebra oriented analysis here.
 </p>
 
 <p>Using our insights about the SVD of the design matrix \( \boldsymbol{X} \) 
 We have already analyzed the OLS solutions in terms of the eigenvectors (the columns) of the right singular value matrix \( \boldsymbol{U} \) as
 </p>
 $$
-\tilde{\boldsymbol{y}}_{\mathrm{OLS}}=\boldsymbol{X}\boldsymbol{\beta}  =\boldsymbol{U}\boldsymbol{U}^T\boldsymbol{y}.
+\tilde{\boldsymbol{y}}_{\mathrm{OLS}}=\boldsymbol{X}\boldsymbol{\theta}  =\boldsymbol{U}\boldsymbol{U}^T\boldsymbol{y}.
 $$
 
 <p>For Ridge regression this becomes</p>
 
 $$
-\tilde{\boldsymbol{y}}_{\mathrm{Ridge}}=\boldsymbol{X}\boldsymbol{\beta}_{\mathrm{Ridge}} = \boldsymbol{U\Sigma V^T}\left(\boldsymbol{V}\boldsymbol{\Sigma}^2\boldsymbol{V}^T+\lambda\boldsymbol{I} \right)^{-1}(\boldsymbol{U\Sigma V^T})^T\boldsymbol{y}=\sum_{j=0}^{p-1}\boldsymbol{u}_j\boldsymbol{u}_j^T\frac{\sigma_j^2}{\sigma_j^2+\lambda}\boldsymbol{y},
+\tilde{\boldsymbol{y}}_{\mathrm{Ridge}}=\boldsymbol{X}\boldsymbol{\theta}_{\mathrm{Ridge}} = \boldsymbol{U\Sigma V^T}\left(\boldsymbol{V}\boldsymbol{\Sigma}^2\boldsymbol{V}^T+\lambda\boldsymbol{I} \right)^{-1}(\boldsymbol{U\Sigma V^T})^T\boldsymbol{y}=\sum_{j=0}^{p-1}\boldsymbol{u}_j\boldsymbol{u}_j^T\frac{\sigma_j^2}{\sigma_j^2+\lambda}\boldsymbol{y},
 $$
 
 <p>with the vectors \( \boldsymbol{u}_j \) being the columns of \( \boldsymbol{U} \) from the SVD of the matrix \( \boldsymbol{X} \). </p>
@@ -2638,13 +2637,13 @@ $$
 
 <p>In this case the standard OLS results in </p>
 $$
-\boldsymbol{\beta}^{\mathrm{OLS}} = \boldsymbol{X}^T\boldsymbol{y}=\sum_{i=0}^{n-1}\boldsymbol{u}_i\boldsymbol{u}_i^T\boldsymbol{y},
+\boldsymbol{\theta}^{\mathrm{OLS}} = \boldsymbol{X}^T\boldsymbol{y}=\sum_{i=0}^{n-1}\boldsymbol{u}_i\boldsymbol{u}_i^T\boldsymbol{y},
 $$
 
 <p>and</p>
 
 $$
-\boldsymbol{\beta}^{\mathrm{Ridge}} = \left(\boldsymbol{I}+\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}=\left(1+\lambda\right)^{-1}\boldsymbol{\beta}^{\mathrm{OLS}},
+\boldsymbol{\theta}^{\mathrm{Ridge}} = \left(\boldsymbol{I}+\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}=\left(1+\lambda\right)^{-1}\boldsymbol{\theta}^{\mathrm{OLS}},
 $$
 
 <p>that is the Ridge estimator scales the OLS estimator by the inverse of a factor \( 1+\lambda \), and
@@ -2664,28 +2663,28 @@ Similarly, <a href="https://arxiv.org/abs/1803.08823" target="_blank">Mehta et a
 <p>Using the matrix-vector expression for Lasso regression, we have the following <b>cost</b> function</p>
 
 $$
-C(\boldsymbol{X},\boldsymbol{\beta})=\frac{1}{n}\left\{(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})^T(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})\right\}+\lambda\vert\vert\boldsymbol{\beta}\vert\vert_1,
+C(\boldsymbol{X},\boldsymbol{\theta})=\frac{1}{n}\left\{(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta})^T(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta})\right\}+\lambda\vert\vert\boldsymbol{\theta}\vert\vert_1,
 $$
 
-<p>Taking the derivative with respect to \( \boldsymbol{\beta} \) and recalling that the derivative of the absolute value is (we drop the boldfaced vector symbol for simplicty)</p>
+<p>Taking the derivative with respect to \( \boldsymbol{\theta} \) and recalling that the derivative of the absolute value is (we drop the boldfaced vector symbol for simplicty)</p>
 $$
-\frac{d \vert \beta\vert}{d \beta}=\mathrm{sgn}(\beta)=\left\{\begin{array}{cc} 1 & \beta > 0 \\-1 & \beta < 0, \end{array}\right.
+\frac{d \vert \theta\vert}{d \theta}=\mathrm{sgn}(\theta)=\left\{\begin{array}{cc} 1 & \theta > 0 \\-1 & \theta < 0, \end{array}\right.
 $$
 
 <p>we have that the derivative of the cost function is</p>
 
 $$
-\frac{\partial C(\boldsymbol{X},\boldsymbol{\beta})}{\partial \boldsymbol{\beta}}=-\frac{2}{n}\boldsymbol{X}^T(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})+\lambda sgn(\boldsymbol{\beta})=0,
+\frac{\partial C(\boldsymbol{X},\boldsymbol{\theta})}{\partial \boldsymbol{\theta}}=-\frac{2}{n}\boldsymbol{X}^T(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta})+\lambda sgn(\boldsymbol{\theta})=0,
 $$
 
 <p>and reordering we have</p>
 $$
-\boldsymbol{X}^T\boldsymbol{X}\boldsymbol{\beta}+\frac{n}{2}\lambda sgn(\boldsymbol{\beta})=2\boldsymbol{X}^T\boldsymbol{y}.
+\boldsymbol{X}^T\boldsymbol{X}\boldsymbol{\theta}+\frac{n}{2}\lambda sgn(\boldsymbol{\theta})=2\boldsymbol{X}^T\boldsymbol{y}.
 $$
 
 <p>We can redefine \( \lambda \) to absorb the constant \( n/2 \) and we rewrite the last equation as</p>
 $$
-\boldsymbol{X}^T\boldsymbol{X}\boldsymbol{\beta}+\lambda sgn(\boldsymbol{\beta})=2\boldsymbol{X}^T\boldsymbol{y}.
+\boldsymbol{X}^T\boldsymbol{X}\boldsymbol{\theta}+\lambda sgn(\boldsymbol{\theta})=2\boldsymbol{X}^T\boldsymbol{y}.
 $$
 
 <p>This equation does not lead to a nice analytical equation as in either Ridge regression or ordinary least squares. This equation can however be solved by using standard convex optimization algorithms using for example the Python package <a href="https://cvxopt.org/" target="_blank">CVXOPT</a>. We will discuss this later. </p>
diff --git a/doc/pub/week35/html/week35.html b/doc/pub/week35/html/week35.html
index 4cefeb9d7..8391c4fdc 100644
--- a/doc/pub/week35/html/week35.html
+++ b/doc/pub/week35/html/week35.html
@@ -380,13 +380,12 @@ MathJax.Hub.Config({
 <!-- !split --><br><br><br><br><br><br><br><br><br><br>
 <h2 id="reminder-from-last-week">Reminder from last week </h2>
 
-<p>We need first a reminder from last week about linear regression. </p>
+<p>We need first a reminder from last week about linear regression. We are going to fit a continuous function with linear parameterization in terms of the parameters  \( \boldsymbol{\theta} \) and our first encounter is ordinary least squares.</p>
 
-<p>Fitting a continuous function with linear parameterization in terms of the parameters  \( \boldsymbol{\beta} \).</p>
 <ul>
-<li> Method of choice for fitting a continuous function!</li>
+<li> It is the method of choice for fitting a continuous function</li>
 <li> Gives an excellent introduction to central Machine Learning features with <b>understandable pedagogical</b> links to other methods like <b>Neural Networks</b>, <b>Support Vector Machines</b> etc</li>
-<li> Analytical expression for the fitting parameters \( \boldsymbol{\beta} \)</li>
+<li> Analytical expression for the fitting parameters \( \boldsymbol{\theta} \)</li>
 <li> Analytical expressions for statistical propertiers like mean values, variances, confidence intervals and more</li>
 <li> Analytical relation with probabilistic interpretations</li> 
 <li> Easy to introduce basic concepts like bias-variance tradeoff, cross-validation, resampling and regularization techniques and many other ML topics</li>
@@ -424,19 +423,19 @@ value and a variance \( \sigma^2 \).
 <p>In linear regression we approximate the unknown function with another
 continuous function \( \tilde{\boldsymbol{y}}(\boldsymbol{x}) \) which depends linearly on
 some unknown parameters
-\( \boldsymbol{\beta}^T=[\beta_0,\beta_1,\beta_2,\dots,\beta_{p-1}] \).
+\( \boldsymbol{\theta}^T=[\theta_0,\theta_1,\theta_2,\dots,\theta_{p-1}] \).
 </p>
 
 <p>Last week we introduced the so-called design matrix in order to define
 the approximation \( \boldsymbol{\tilde{y}} \) via the unknown quantity
-\( \boldsymbol{\beta} \) as
+\( \boldsymbol{\theta} \) as
 </p>
 
 $$
-\boldsymbol{\tilde{y}}= \boldsymbol{X}\boldsymbol{\beta},
+\boldsymbol{\tilde{y}}= \boldsymbol{X}\boldsymbol{\theta},
 $$
 
-<p>and in order to find the optimal parameters \( \beta_i \) we defined a function which
+<p>and in order to find the optimal parameters \( \theta_i \) we defined a function which
 gives a measure of the spread between the values \( y_i \) (which
 represent the output values we want to reproduce) and the parametrized
 values \( \tilde{y}_i \), namely the so-called cost/loss function.
@@ -447,12 +446,12 @@ values \( \tilde{y}_i \), namely the so-called cost/loss function.
 
 <p>We used the mean squared error to define the way we measure the quality of our model</p>
 $$
-C(\boldsymbol{\beta})=\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)^T\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)\right\},
+C(\boldsymbol{\theta})=\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)^T\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)\right\},
 $$
 
 <p>or using the matrix \( \boldsymbol{X} \) and in a more compact matrix-vector notation as</p>
 $$
-C(\boldsymbol{\beta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right\}.
+C(\boldsymbol{\theta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right\}.
 $$
 
 <p>This function represents one of many possible ways to define the so-called cost function.</p>
@@ -462,24 +461,24 @@ the function \( C \) as
 </p>
 
 $$
-C(\boldsymbol{\beta})=\frac{1}{2n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2,
+C(\boldsymbol{\theta})=\frac{1}{2n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2,
 $$
 
-<p>since when taking the first derivative with respect to the unknown parameters \( \beta \), the factor of \( 2 \) cancels out. </p>
+<p>since when taking the first derivative with respect to the unknown parameters \( \theta \), the factor of \( 2 \) cancels out. </p>
 
 <!-- !split --><br><br><br><br><br><br><br><br><br><br>
 <h2 id="interpretations-and-optimizing-our-parameters">Interpretations and optimizing our parameters  </h2>
 
 <p>The function </p>
 $$
-C(\boldsymbol{\beta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right\},
+C(\boldsymbol{\theta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right\},
 $$
 
 <p>can be linked to the variance of the quantity \( y_i \) if we interpret the latter as the mean value. 
 When linking (see the discussions next week) with the maximum likelihood approach below, we will indeed interpret \( y_i \) as a mean value
 </p>
 $$
-y_{i}=\langle y_i \rangle = \beta_0x_{i,0}+\beta_1x_{i,1}+\beta_2x_{i,2}+\dots+\beta_{n-1}x_{i,n-1}+\epsilon_i,
+y_{i}=\langle y_i \rangle = \theta_0x_{i,0}+\theta_1x_{i,1}+\theta_2x_{i,2}+\dots+\theta_{n-1}x_{i,n-1}+\epsilon_i,
 $$
 
 <p>where \( \langle y_i \rangle \) is the mean value. Keep in mind also that
@@ -492,25 +491,25 @@ the standard deviation discussed earlier. In the discussion here we
 will treat \( y_i \) as our exact value for the response variable.
 </p>
 
-<p>In order to find the parameters \( \beta_i \) we will then minimize the spread of \( C(\boldsymbol{\beta}) \), that is we are going to solve the problem</p>
+<p>In order to find the parameters \( \theta_i \) we will then minimize the spread of \( C(\boldsymbol{\theta}) \), that is we are going to solve the problem</p>
 $$
-{\displaystyle \min_{\boldsymbol{\beta}\in
-{\mathbb{R}}^{p}}}\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right\}.
+{\displaystyle \min_{\boldsymbol{\theta}\in
+{\mathbb{R}}^{p}}}\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right\}.
 $$
 
 <p>In practical terms it means we will require</p>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \beta_j} = \frac{\partial }{\partial \beta_j}\left[ \frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\beta_0x_{i,0}-\beta_1x_{i,1}-\beta_2x_{i,2}-\dots-\beta_{n-1}x_{i,n-1}\right)^2\right]=0, 
+\frac{\partial C(\boldsymbol{\theta})}{\partial \theta_j} = \frac{\partial }{\partial \theta_j}\left[ \frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\theta_0x_{i,0}-\theta_1x_{i,1}-\theta_2x_{i,2}-\dots-\theta_{n-1}x_{i,n-1}\right)^2\right]=0, 
 $$
 
 <p>which results in</p>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \beta_j} = -\frac{2}{n}\left[ \sum_{i=0}^{n-1}x_{ij}\left(y_i-\beta_0x_{i,0}-\beta_1x_{i,1}-\beta_2x_{i,2}-\dots-\beta_{n-1}x_{i,n-1}\right)\right]=0, 
+\frac{\partial C(\boldsymbol{\theta})}{\partial \theta_j} = -\frac{2}{n}\left[ \sum_{i=0}^{n-1}x_{ij}\left(y_i-\theta_0x_{i,0}-\theta_1x_{i,1}-\theta_2x_{i,2}-\dots-\theta_{n-1}x_{i,n-1}\right)\right]=0, 
 $$
 
 <p>or in a matrix-vector form as (multiplying away the factor \( -2/n \), see derivation below)</p>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}^T} = 0 = \boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right).  
+\frac{\partial C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}^T} = 0 = \boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right).  
 $$
 
 
@@ -521,17 +520,17 @@ $$
 <p>
 <p>We can rewrite, see the derivations below, </p>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}^T} = 0 = \boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right),  
+\frac{\partial C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}^T} = 0 = \boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right),  
 $$
 
 <p>as</p>
 $$
-\boldsymbol{X}^T\boldsymbol{y} = \boldsymbol{X}^T\boldsymbol{X}\boldsymbol{\beta},  
+\boldsymbol{X}^T\boldsymbol{y} = \boldsymbol{X}^T\boldsymbol{X}\boldsymbol{\theta},  
 $$
 
 <p>and if the matrix \( \boldsymbol{X}^T\boldsymbol{X} \) is invertible we have the solution</p>
 $$
-\boldsymbol{\beta} =\left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
+\boldsymbol{\theta} =\left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
 $$
 
 <p>We note also that since our design matrix is defined as \( \boldsymbol{X}\in
@@ -739,46 +738,46 @@ $$
 
 <p>We defined earlier a possible cost function using the mean squared error</p>
 $$
-C(\boldsymbol{\beta})=\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)^T\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)\right\},
+C(\boldsymbol{\theta})=\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)^T\left(\boldsymbol{y}-\boldsymbol{\tilde{y}}\right)\right\},
 $$
 
 <p>or using the design/feature matrix \( \boldsymbol{X} \) we have the more compact matrix-vector</p>
 $$
-C(\boldsymbol{\beta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right\}.
+C(\boldsymbol{\theta})=\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right\}.
 $$
 
-<p>We note that the design matrix \( \boldsymbol{X} \) does not depend on the unknown parameters defined by the vector \( \boldsymbol{\beta} \).
-We are now interested in minimizing the cost function with respect to the unknown parameters \( \boldsymbol{\beta} \).
+<p>We note that the design matrix \( \boldsymbol{X} \) does not depend on the unknown parameters defined by the vector \( \boldsymbol{\theta} \).
+We are now interested in minimizing the cost function with respect to the unknown parameters \( \boldsymbol{\theta} \).
 </p>
 
 <p>The mean squared error is a scalar and if we use the results from example three above, we can define a new vector</p>
 $$
-\boldsymbol{w}=\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta},
+\boldsymbol{w}=\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta},
 $$
 
-<p>which depends on \( \boldsymbol{\beta} \). We rewrite the cost function as</p>
+<p>which depends on \( \boldsymbol{\theta} \). We rewrite the cost function as</p>
 $$
-C(\boldsymbol{\beta})=\frac{1}{n}\boldsymbol{w}^T\boldsymbol{w},
+C(\boldsymbol{\theta})=\frac{1}{n}\boldsymbol{w}^T\boldsymbol{w},
 $$
 
 <p>with partial derivative</p>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}}=\frac{2}{n}\boldsymbol{w}^T\frac{\partial \boldsymbol{w}}{\partial \boldsymbol{\beta}},
+\frac{\partial C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}}=\frac{2}{n}\boldsymbol{w}^T\frac{\partial \boldsymbol{w}}{\partial \boldsymbol{\theta}},
 $$
 
 <p>and using that</p>
 $$
-\frac{\partial \boldsymbol{w}}{\partial \boldsymbol{\beta}}=-\boldsymbol{X},
+\frac{\partial \boldsymbol{w}}{\partial \boldsymbol{\theta}}=-\boldsymbol{X},
 $$
 
 <p>where we used the result from example two above. Inserting the last expression we obtain</p>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}}=-\frac{2}{n}\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\boldsymbol{X},
+\frac{\partial C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}}=-\frac{2}{n}\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\boldsymbol{X},
 $$
 
 <p>or as</p>
 $$
-\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}^T}=-\frac{2}{n}\boldsymbol{X}^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right).
+\frac{\partial C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}^T}=-\frac{2}{n}\boldsymbol{X}^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right).
 $$
 
 
@@ -809,13 +808,13 @@ $$
 
 <p>A very important matrix we will meet again and again in machine
 learning is the Hessian.  It is given by the second derivative of the
-cost function with respect to the parameters \( \boldsymbol{\beta} \). Using the above
+cost function with respect to the parameters \( \boldsymbol{\theta} \). Using the above
 expression for derivatives of vectors and matrices, we find that the
 second derivative of the mean squared error as cost function is,
 </p>
 
 $$
-\frac{\partial}{\partial \boldsymbol{\beta}}\frac{\partial C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}^T} =\frac{\partial}{\partial \boldsymbol{\beta}}\left[-\frac{2}{n}\boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right]=\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}.  
+\frac{\partial}{\partial \boldsymbol{\theta}}\frac{\partial C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}^T} =\frac{\partial}{\partial \boldsymbol{\theta}}\left[-\frac{2}{n}\boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right]=\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}.  
 $$
 
 <p>The Hessian matrix plays an important role and is defined here as</p>
@@ -826,7 +825,7 @@ $$
 
 <p>For ordinary least squares, it is inversely proportional (derivation
 next week) with the variance of the optimal parameters
-\( \hat{\boldsymbol{\beta}} \). Furthermore, we will see later this week that it is
+\( \hat{\boldsymbol{\theta}} \). Furthermore, we will see later this week that it is
 (aside the factor \( 1/n \)) equal to the covariance matrix. It plays also a very
 important role in optmization algorithms and Principal Component
 Analysis as a way to reduce the dimensionality of a machine learning/data analysis
@@ -843,20 +842,20 @@ v
 <p>
 <p>The residuals \( \boldsymbol{\epsilon} \) are in turn given by</p>
 $$
-\boldsymbol{\epsilon} = \boldsymbol{y}-\boldsymbol{\tilde{y}} = \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta},
+\boldsymbol{\epsilon} = \boldsymbol{y}-\boldsymbol{\tilde{y}} = \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta},
 $$
 
 <p>and with </p>
 $$
-\boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)= 0, 
+\boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)= 0, 
 $$
 
 <p>we have</p>
 $$
-\boldsymbol{X}^T\boldsymbol{\epsilon}=\boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)= 0, 
+\boldsymbol{X}^T\boldsymbol{\epsilon}=\boldsymbol{X}^T\left( \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)= 0, 
 $$
 
-<p>meaning that the solution for \( \boldsymbol{\beta} \) is the one which minimizes the residuals.  </p>
+<p>meaning that the solution for \( \boldsymbol{\theta} \) is the one which minimizes the residuals.  </p>
 </div>
 
 
@@ -868,17 +867,17 @@ we condiser a simple polynomial fit.
 We assume our data can represented by a fourth-order polynomial. For the $i$th component we have 
 </p>
 $$
-\tilde{y}_i = \beta_0+\beta_1x_i+\beta_2x_i^2+\beta_3x_i^3+\beta_4x_i^4.
+\tilde{y}_i = \theta_0+\theta_1x_i+\theta_2x_i^2+\theta_3x_i^3+\theta_4x_i^4.
 $$
 
-<p>we have five predictors/features. The first is the intercept \( \beta_0 \). The other terms are \( \beta_i \) with \( i=1,2,3,4 \). Furthermore we have \( n \) entries for each predictor. It means that our design matrix is an 
+<p>we have five predictors/features. The first is the intercept \( \theta_0 \). The other terms are \( \theta_i \) with \( i=1,2,3,4 \). Furthermore we have \( n \) entries for each predictor. It means that our design matrix is an 
 \( n\times p \) matrix \( \boldsymbol{X} \).
 </p>
 
 <!-- !split --><br><br><br><br><br><br><br><br><br><br>
 <h2 id="own-code-for-ordinary-least-squares">Own code for Ordinary Least Squares </h2>
 
-<p>It is rather straightforward to implement the matrix inversion and obtain the parameters \( \boldsymbol{\beta} \). After having defined the matrix \( \boldsymbol{X} \) and the outputs \( \boldsymbol{y} \) we have </p>
+<p>It is rather straightforward to implement the matrix inversion and obtain the parameters \( \boldsymbol{\theta} \). After having defined the matrix \( \boldsymbol{X} \) and the outputs \( \boldsymbol{y} \) we have </p>
 
 <!-- code=python (!bc pycod) typeset with pygments style "default" -->
 <div class="cell border-box-sizing code_cell rendered">
@@ -1468,18 +1467,18 @@ plt<span style="color: #666666">.</span>show()
 
 <p>What is presented here is a mathematical analysis of various regression algorithms (ordinary least  squares, Ridge and Lasso Regression). The analysis is based on an important algorithm in linear algebra, the so-called Singular Value Decomposition (SVD). </p>
 
-<p>We have shown that in ordinary least squares the optimal parameters \( \beta \) are given by</p>
+<p>We have shown that in ordinary least squares the optimal parameters \( \theta \) are given by</p>
 
 $$
-\hat{\boldsymbol{\beta}} = \left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
+\hat{\boldsymbol{\theta}} = \left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
 $$
 
-<p>The <b>hat</b> over \( \boldsymbol{\beta} \) means we have the optimal parameters after minimization of the cost function.</p>
+<p>The <b>hat</b> over \( \boldsymbol{\theta} \) means we have the optimal parameters after minimization of the cost function.</p>
 
 <p>This means that our best model is defined as</p>
 
 $$
-\tilde{\boldsymbol{y}}=\boldsymbol{X}\hat{\boldsymbol{\beta}} = \boldsymbol{X}\left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
+\tilde{\boldsymbol{y}}=\boldsymbol{X}\hat{\boldsymbol{\theta}} = \boldsymbol{X}\left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
 $$
 
 <p>We now define a matrix</p>
@@ -1489,7 +1488,7 @@ $$
 
 <p>We can rewrite</p>
 $$
-\tilde{\boldsymbol{y}}=\boldsymbol{X}\hat{\boldsymbol{\beta}} = \boldsymbol{A}\boldsymbol{y}.
+\tilde{\boldsymbol{y}}=\boldsymbol{X}\hat{\boldsymbol{\theta}} = \boldsymbol{A}\boldsymbol{y}.
 $$
 
 <p>The matrix \( \boldsymbol{A} \) has the important property that \( \boldsymbol{A}^2=\boldsymbol{A} \). This is the definition of a projection matrix.
@@ -1627,16 +1626,16 @@ This is equivalent to saying that the matrix \( \boldsymbol{X} \) has at least a
 <p>If our design matrix \( \boldsymbol{X} \) which enters the linear regression problem</p>
 $$
 \begin{align}
-\boldsymbol{\beta} & =  (\boldsymbol{X}^{T} \boldsymbol{X})^{-1} \boldsymbol{X}^{T} \boldsymbol{y},
+\boldsymbol{\theta} & =  (\boldsymbol{X}^{T} \boldsymbol{X})^{-1} \boldsymbol{X}^{T} \boldsymbol{y},
 \label{_auto1}
 \end{align}
 $$
 
 <p>has linearly dependent column vectors, we will not be able to compute the inverse
-of \( \boldsymbol{X}^T\boldsymbol{X} \) and we cannot find the parameters (estimators) \( \beta_i \). 
+of \( \boldsymbol{X}^T\boldsymbol{X} \) and we cannot find the parameters (estimators) \( \theta_i \). 
 The estimators are only well-defined if \( (\boldsymbol{X}^{T}\boldsymbol{X})^{-1} \) exits. 
 This is more likely to happen when the matrix \( \boldsymbol{X} \) is high-dimensional. In this case it is likely to encounter a situation where 
-the regression parameters \( \beta_i \) cannot be estimated.
+the regression parameters \( \theta_i \) cannot be estimated.
 </p>
 
 <p>A cheap  <em>ad hoc</em> approach is  simply to add a small diagonal component to the matrix to invert, that is we change</p>
@@ -2033,7 +2032,7 @@ function, that is we have
 </p>
 
 $$
-\frac{\partial^2 C(\boldsymbol{\beta})}{\partial \boldsymbol{\beta}\partial \boldsymbol{\beta}^T} =\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}.  
+\frac{\partial^2 C(\boldsymbol{\theta})}{\partial \boldsymbol{\theta}\partial \boldsymbol{\theta}^T} =\frac{2}{n}\boldsymbol{X}^T\boldsymbol{X}.  
 $$
 
 <p>This quantity defines was what is called the Hessian matrix (the second derivative of a function we want to optimize).</p>
@@ -2575,13 +2574,13 @@ values and the column vectors of \( \boldsymbol{V} \).
 our optimization problem is
 </p>
 $$
-{\displaystyle \min_{\boldsymbol{\beta}\in {\mathbb{R}}^{p}}}\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right)\right\}.
+{\displaystyle \min_{\boldsymbol{\theta}\in {\mathbb{R}}^{p}}}\frac{1}{n}\left\{\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\right)\right\}.
 $$
 
 <p>or we can state it as</p>
 $$
-{\displaystyle \min_{\boldsymbol{\beta}\in
-{\mathbb{R}}^{p}}}\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\vert\vert_2^2,
+{\displaystyle \min_{\boldsymbol{\theta}\in
+{\mathbb{R}}^{p}}}\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\vert\vert_2^2,
 $$
 
 <p>where we have used the definition of  a norm-2 vector, that is</p>
@@ -2590,29 +2589,29 @@ $$
 $$
 
 <p>By minimizing the above equation with respect to the parameters
-\( \boldsymbol{\beta} \) we could then obtain an analytical expression for the
-parameters \( \boldsymbol{\beta} \).  We can add a regularization parameter \( \lambda \) by
+\( \boldsymbol{\theta} \) we could then obtain an analytical expression for the
+parameters \( \boldsymbol{\theta} \).  We can add a regularization parameter \( \lambda \) by
 defining a new cost function to be optimized, that is
 </p>
 
 $$
-{\displaystyle \min_{\boldsymbol{\beta}\in
-{\mathbb{R}}^{p}}}\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\beta}\vert\vert_2^2
+{\displaystyle \min_{\boldsymbol{\theta}\in
+{\mathbb{R}}^{p}}}\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\theta}\vert\vert_2^2
 $$
 
 <p>which leads to the Ridge regression minimization problem where we
-require that \( \vert\vert \boldsymbol{\beta}\vert\vert_2^2\le t \), where \( t \) is
+require that \( \vert\vert \boldsymbol{\theta}\vert\vert_2^2\le t \), where \( t \) is
 a finite number larger than zero. By defining
 </p>
 
 $$
-C(\boldsymbol{X},\boldsymbol{\beta})=\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\beta}\vert\vert_1,
+C(\boldsymbol{X},\boldsymbol{\theta})=\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\theta}\vert\vert_1,
 $$
 
 <p>we have a new optimization equation</p>
 $$
-{\displaystyle \min_{\boldsymbol{\beta}\in
-{\mathbb{R}}^{p}}}\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\beta}\vert\vert_1
+{\displaystyle \min_{\boldsymbol{\theta}\in
+{\mathbb{R}}^{p}}}\frac{1}{n}\vert\vert \boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta}\vert\vert_2^2+\lambda\vert\vert \boldsymbol{\theta}\vert\vert_1
 $$
 
 <p>which leads to Lasso regression. Lasso stands for least absolute shrinkage and selection operator. </p>
@@ -2629,37 +2628,37 @@ $$
 <p>Using the matrix-vector expression for Ridge regression and dropping the parameter \( 1/n \) in front of the standard means squared error equation, we have</p>
 
 $$
-C(\boldsymbol{X},\boldsymbol{\beta})=\left\{(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})^T(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})\right\}+\lambda\boldsymbol{\beta}^T\boldsymbol{\beta},
+C(\boldsymbol{X},\boldsymbol{\theta})=\left\{(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta})^T(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta})\right\}+\lambda\boldsymbol{\theta}^T\boldsymbol{\theta},
 $$
 
 <p>and 
-taking the derivatives with respect to \( \boldsymbol{\beta} \) we obtain then
+taking the derivatives with respect to \( \boldsymbol{\theta} \) we obtain then
 a slightly modified matrix inversion problem which for finite values
 of \( \lambda \) does not suffer from singularity problems. We obtain
 the optimal parameters
 </p>
 $$
-\hat{\boldsymbol{\beta}}_{\mathrm{Ridge}} = \left(\boldsymbol{X}^T\boldsymbol{X}+\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y},
+\hat{\boldsymbol{\theta}}_{\mathrm{Ridge}} = \left(\boldsymbol{X}^T\boldsymbol{X}+\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y},
 $$
 
 <p>with \( \boldsymbol{I} \) being a \( p\times p \) identity matrix with the constraint that</p>
 
 $$
-\sum_{i=0}^{p-1} \beta_i^2 \leq t,
+\sum_{i=0}^{p-1} \theta_i^2 \leq t,
 $$
 
 <p>with \( t \) a finite positive number. </p>
 
-<p>If we keep the \( 1/n \) factor, the equation for the optimal \( \beta \) changes to</p>
+<p>If we keep the \( 1/n \) factor, the equation for the optimal \( \theta \) changes to</p>
 $$
-\hat{\boldsymbol{\beta}}_{\mathrm{Ridge}} = \left(\boldsymbol{X}^T\boldsymbol{X}+n\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
+\hat{\boldsymbol{\theta}}_{\mathrm{Ridge}} = \left(\boldsymbol{X}^T\boldsymbol{X}+n\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}.
 $$
 
 <p>In many textbooks the \( 1/n \) term is often omitted. Note that a library like <b>Scikit-Learn</b> does not include the \( 1/n \) factor in the setup of the cost function.</p>
 
 <p>When we compare this with the ordinary least squares result we have</p>
 $$
-\hat{\boldsymbol{\beta}}_{\mathrm{OLS}} = \left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y},
+\hat{\boldsymbol{\theta}}_{\mathrm{OLS}} = \left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y},
 $$
 
 <p>which can lead to singular matrices. However, with the SVD, we can always compute the inverse of the matrix \( \boldsymbol{X}^T\boldsymbol{X} \).</p>
@@ -2668,20 +2667,20 @@ $$
 modified diagonal term added to \( \boldsymbol{X}^T\boldsymbol{X} \). The consequences, in
 particular for our discussion of the bias-variance tradeoff are rather
 interesting. We will see that for specific values of \( \lambda \), we may
-even reduce the variance of the optimal parameters \( \boldsymbol{\beta} \). These topics and other related ones, will be discussed after the more linear algebra oriented analysis here.
+even reduce the variance of the optimal parameters \( \boldsymbol{\theta} \). These topics and other related ones, will be discussed after the more linear algebra oriented analysis here.
 </p>
 
 <p>Using our insights about the SVD of the design matrix \( \boldsymbol{X} \) 
 We have already analyzed the OLS solutions in terms of the eigenvectors (the columns) of the right singular value matrix \( \boldsymbol{U} \) as
 </p>
 $$
-\tilde{\boldsymbol{y}}_{\mathrm{OLS}}=\boldsymbol{X}\boldsymbol{\beta}  =\boldsymbol{U}\boldsymbol{U}^T\boldsymbol{y}.
+\tilde{\boldsymbol{y}}_{\mathrm{OLS}}=\boldsymbol{X}\boldsymbol{\theta}  =\boldsymbol{U}\boldsymbol{U}^T\boldsymbol{y}.
 $$
 
 <p>For Ridge regression this becomes</p>
 
 $$
-\tilde{\boldsymbol{y}}_{\mathrm{Ridge}}=\boldsymbol{X}\boldsymbol{\beta}_{\mathrm{Ridge}} = \boldsymbol{U\Sigma V^T}\left(\boldsymbol{V}\boldsymbol{\Sigma}^2\boldsymbol{V}^T+\lambda\boldsymbol{I} \right)^{-1}(\boldsymbol{U\Sigma V^T})^T\boldsymbol{y}=\sum_{j=0}^{p-1}\boldsymbol{u}_j\boldsymbol{u}_j^T\frac{\sigma_j^2}{\sigma_j^2+\lambda}\boldsymbol{y},
+\tilde{\boldsymbol{y}}_{\mathrm{Ridge}}=\boldsymbol{X}\boldsymbol{\theta}_{\mathrm{Ridge}} = \boldsymbol{U\Sigma V^T}\left(\boldsymbol{V}\boldsymbol{\Sigma}^2\boldsymbol{V}^T+\lambda\boldsymbol{I} \right)^{-1}(\boldsymbol{U\Sigma V^T})^T\boldsymbol{y}=\sum_{j=0}^{p-1}\boldsymbol{u}_j\boldsymbol{u}_j^T\frac{\sigma_j^2}{\sigma_j^2+\lambda}\boldsymbol{y},
 $$
 
 <p>with the vectors \( \boldsymbol{u}_j \) being the columns of \( \boldsymbol{U} \) from the SVD of the matrix \( \boldsymbol{X} \). </p>
@@ -2715,13 +2714,13 @@ $$
 
 <p>In this case the standard OLS results in </p>
 $$
-\boldsymbol{\beta}^{\mathrm{OLS}} = \boldsymbol{X}^T\boldsymbol{y}=\sum_{i=0}^{n-1}\boldsymbol{u}_i\boldsymbol{u}_i^T\boldsymbol{y},
+\boldsymbol{\theta}^{\mathrm{OLS}} = \boldsymbol{X}^T\boldsymbol{y}=\sum_{i=0}^{n-1}\boldsymbol{u}_i\boldsymbol{u}_i^T\boldsymbol{y},
 $$
 
 <p>and</p>
 
 $$
-\boldsymbol{\beta}^{\mathrm{Ridge}} = \left(\boldsymbol{I}+\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}=\left(1+\lambda\right)^{-1}\boldsymbol{\beta}^{\mathrm{OLS}},
+\boldsymbol{\theta}^{\mathrm{Ridge}} = \left(\boldsymbol{I}+\lambda\boldsymbol{I}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}=\left(1+\lambda\right)^{-1}\boldsymbol{\theta}^{\mathrm{OLS}},
 $$
 
 <p>that is the Ridge estimator scales the OLS estimator by the inverse of a factor \( 1+\lambda \), and
@@ -2741,28 +2740,28 @@ Similarly, <a href="https://arxiv.org/abs/1803.08823" target="_blank">Mehta et a
 <p>Using the matrix-vector expression for Lasso regression, we have the following <b>cost</b> function</p>
 
 $$
-C(\boldsymbol{X},\boldsymbol{\beta})=\frac{1}{n}\left\{(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})^T(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})\right\}+\lambda\vert\vert\boldsymbol{\beta}\vert\vert_1,
+C(\boldsymbol{X},\boldsymbol{\theta})=\frac{1}{n}\left\{(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta})^T(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta})\right\}+\lambda\vert\vert\boldsymbol{\theta}\vert\vert_1,
 $$
 
-<p>Taking the derivative with respect to \( \boldsymbol{\beta} \) and recalling that the derivative of the absolute value is (we drop the boldfaced vector symbol for simplicty)</p>
+<p>Taking the derivative with respect to \( \boldsymbol{\theta} \) and recalling that the derivative of the absolute value is (we drop the boldfaced vector symbol for simplicty)</p>
 $$
-\frac{d \vert \beta\vert}{d \beta}=\mathrm{sgn}(\beta)=\left\{\begin{array}{cc} 1 & \beta > 0 \\-1 & \beta < 0, \end{array}\right.
+\frac{d \vert \theta\vert}{d \theta}=\mathrm{sgn}(\theta)=\left\{\begin{array}{cc} 1 & \theta > 0 \\-1 & \theta < 0, \end{array}\right.
 $$
 
 <p>we have that the derivative of the cost function is</p>
 
 $$
-\frac{\partial C(\boldsymbol{X},\boldsymbol{\beta})}{\partial \boldsymbol{\beta}}=-\frac{2}{n}\boldsymbol{X}^T(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})+\lambda sgn(\boldsymbol{\beta})=0,
+\frac{\partial C(\boldsymbol{X},\boldsymbol{\theta})}{\partial \boldsymbol{\theta}}=-\frac{2}{n}\boldsymbol{X}^T(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\theta})+\lambda sgn(\boldsymbol{\theta})=0,
 $$
 
 <p>and reordering we have</p>
 $$
-\boldsymbol{X}^T\boldsymbol{X}\boldsymbol{\beta}+\frac{n}{2}\lambda sgn(\boldsymbol{\beta})=2\boldsymbol{X}^T\boldsymbol{y}.
+\boldsymbol{X}^T\boldsymbol{X}\boldsymbol{\theta}+\frac{n}{2}\lambda sgn(\boldsymbol{\theta})=2\boldsymbol{X}^T\boldsymbol{y}.
 $$
 
 <p>We can redefine \( \lambda \) to absorb the constant \( n/2 \) and we rewrite the last equation as</p>
 $$
-\boldsymbol{X}^T\boldsymbol{X}\boldsymbol{\beta}+\lambda sgn(\boldsymbol{\beta})=2\boldsymbol{X}^T\boldsymbol{y}.
+\boldsymbol{X}^T\boldsymbol{X}\boldsymbol{\theta}+\lambda sgn(\boldsymbol{\theta})=2\boldsymbol{X}^T\boldsymbol{y}.
 $$
 
 <p>This equation does not lead to a nice analytical equation as in either Ridge regression or ordinary least squares. This equation can however be solved by using standard convex optimization algorithms using for example the Python package <a href="https://cvxopt.org/" target="_blank">CVXOPT</a>. We will discuss this later. </p>
diff --git a/doc/pub/week35/ipynb/ipynb-week35-src.tar.gz b/doc/pub/week35/ipynb/ipynb-week35-src.tar.gz
index bdcd9a5056cab139878d23b8a29a23212e4b9868..61a745084f129d6f5c72e9aaeea9c4bd6b976978 100644
GIT binary patch
literal 192
zcmV;x06+g9iwFR#cB^Or1MSbv3c@f92k@Qu6nTQtuIqLd^x#1d@dY}TxjNU*wnO*!
z?gR9sco`z}cli?%LUP!w*1JvQ?k-piAtX%_7&A-xlq8<(38fq;Cp=Cw7IOxKGR{~4
zWVw@GI_rcPPHCz$DvRpf+%Q&_ANI_zz%&2Ev62?H``%Srfzl3ht!ua;){IB8?Ntti
uMmx5^;I)$mL8u-?QAj7X5|^+w`ea07qwv?yc%J8ZUwZ(vOp?R^2mk;jHd=51

literal 192
zcmV;x06+g9iwFP$b*pFq1MSaC3c@fD2H>uHia9|^(xzPtx^N+gc!88oZLCddlA^u6
zeSoeMH${Yen}0%vVdk(|t#_Nq-CeL4LP(k<FlLtWDG8h36G}NyPB=?5mJkMn#zP$d
zS#G75&N^X+RhsII%A$JLH?)=IhdJ{p@XSAPtfYnQzIT;YptQqW>l$u|b<88#_9}-$
uqaDA%;I)$mL8u-?QAj7X5|^+w`ea07qwv?qc%J8ZUwZ)U`Q-)x2mk<6pjlJ^

diff --git a/doc/pub/week35/ipynb/week35.ipynb b/doc/pub/week35/ipynb/week35.ipynb
index 95e79be49..36dd263dd 100644
--- a/doc/pub/week35/ipynb/week35.ipynb
+++ b/doc/pub/week35/ipynb/week35.ipynb
@@ -2,7 +2,7 @@
  "cells": [
   {
    "cell_type": "markdown",
-   "id": "f59bef82",
+   "id": "3803481e",
    "metadata": {
     "editable": true
    },
@@ -14,7 +14,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "d23d7811",
+   "id": "90e0b055",
    "metadata": {
     "editable": true
    },
@@ -27,7 +27,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "82fad38e",
+   "id": "0b94b0fb",
    "metadata": {
     "editable": true
    },
@@ -49,7 +49,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "380a8031",
+   "id": "c8ad9a9b",
    "metadata": {
     "editable": true
    },
@@ -69,21 +69,20 @@
   },
   {
    "cell_type": "markdown",
-   "id": "fc0a0056",
+   "id": "f5f74962",
    "metadata": {
     "editable": true
    },
    "source": [
     "## Reminder from last week\n",
     "\n",
-    "We need first a reminder from last week about linear regression. \n",
+    "We need first a reminder from last week about linear regression. We are going to fit a continuous function with linear parameterization in terms of the parameters  $\\boldsymbol{\\theta}$ and our first encounter is ordinary least squares.\n",
     "\n",
-    "Fitting a continuous function with linear parameterization in terms of the parameters  $\\boldsymbol{\\beta}$.\n",
-    "* Method of choice for fitting a continuous function!\n",
+    "* It is the method of choice for fitting a continuous function\n",
     "\n",
     "* Gives an excellent introduction to central Machine Learning features with **understandable pedagogical** links to other methods like **Neural Networks**, **Support Vector Machines** etc\n",
     "\n",
-    "* Analytical expression for the fitting parameters $\\boldsymbol{\\beta}$\n",
+    "* Analytical expression for the fitting parameters $\\boldsymbol{\\theta}$\n",
     "\n",
     "* Analytical expressions for statistical propertiers like mean values, variances, confidence intervals and more\n",
     "\n",
@@ -103,7 +102,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "5ff8abac",
+   "id": "d61252cd",
    "metadata": {
     "editable": true
    },
@@ -119,7 +118,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "df0716ae",
+   "id": "3dd45959",
    "metadata": {
     "editable": true
    },
@@ -131,7 +130,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "72d9a463",
+   "id": "0ab40f7c",
    "metadata": {
     "editable": true
    },
@@ -141,7 +140,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "c81a5718",
+   "id": "d219bba3",
    "metadata": {
     "editable": true
    },
@@ -153,7 +152,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "4d357994",
+   "id": "806e921f",
    "metadata": {
     "editable": true
    },
@@ -165,33 +164,33 @@
     "In linear regression we approximate the unknown function with another\n",
     "continuous function $\\tilde{\\boldsymbol{y}}(\\boldsymbol{x})$ which depends linearly on\n",
     "some unknown parameters\n",
-    "$\\boldsymbol{\\beta}^T=[\\beta_0,\\beta_1,\\beta_2,\\dots,\\beta_{p-1}]$.\n",
+    "$\\boldsymbol{\\theta}^T=[\\theta_0,\\theta_1,\\theta_2,\\dots,\\theta_{p-1}]$.\n",
     "\n",
     "Last week we introduced the so-called design matrix in order to define\n",
     "the approximation $\\boldsymbol{\\tilde{y}}$ via the unknown quantity\n",
-    "$\\boldsymbol{\\beta}$ as"
+    "$\\boldsymbol{\\theta}$ as"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "cf4d9cee",
+   "id": "5a10e185",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\boldsymbol{\\tilde{y}}= \\boldsymbol{X}\\boldsymbol{\\beta},\n",
+    "\\boldsymbol{\\tilde{y}}= \\boldsymbol{X}\\boldsymbol{\\theta},\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "c5ad4e38",
+   "id": "87b9af07",
    "metadata": {
     "editable": true
    },
    "source": [
-    "and in order to find the optimal parameters $\\beta_i$ we defined a function which\n",
+    "and in order to find the optimal parameters $\\theta_i$ we defined a function which\n",
     "gives a measure of the spread between the values $y_i$ (which\n",
     "represent the output values we want to reproduce) and the parametrized\n",
     "values $\\tilde{y}_i$, namely the so-called cost/loss function."
@@ -199,7 +198,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "297ced52",
+   "id": "b5e5e170",
    "metadata": {
     "editable": true
    },
@@ -211,19 +210,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "0f7fefee",
+   "id": "bacf306d",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "C(\\boldsymbol{\\beta})=\\frac{1}{n}\\sum_{i=0}^{n-1}\\left(y_i-\\tilde{y}_i\\right)^2=\\frac{1}{n}\\left\\{\\left(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}}\\right)\\right\\},\n",
+    "C(\\boldsymbol{\\theta})=\\frac{1}{n}\\sum_{i=0}^{n-1}\\left(y_i-\\tilde{y}_i\\right)^2=\\frac{1}{n}\\left\\{\\left(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}}\\right)\\right\\},\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "bd8dc4f2",
+   "id": "971edd52",
    "metadata": {
     "editable": true
    },
@@ -233,19 +232,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "da82ea39",
+   "id": "1e764eaf",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "C(\\boldsymbol{\\beta})=\\frac{1}{n}\\left\\{\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)\\right\\}.\n",
+    "C(\\boldsymbol{\\theta})=\\frac{1}{n}\\left\\{\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\right)\\right\\}.\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "23e19e04",
+   "id": "f2f1e2f4",
    "metadata": {
     "editable": true
    },
@@ -258,29 +257,29 @@
   },
   {
    "cell_type": "markdown",
-   "id": "24f8e2c6",
+   "id": "f00007b0",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "C(\\boldsymbol{\\beta})=\\frac{1}{2n}\\sum_{i=0}^{n-1}\\left(y_i-\\tilde{y}_i\\right)^2,\n",
+    "C(\\boldsymbol{\\theta})=\\frac{1}{2n}\\sum_{i=0}^{n-1}\\left(y_i-\\tilde{y}_i\\right)^2,\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "2f9e0a7f",
+   "id": "ad127894",
    "metadata": {
     "editable": true
    },
    "source": [
-    "since when taking the first derivative with respect to the unknown parameters $\\beta$, the factor of $2$ cancels out."
+    "since when taking the first derivative with respect to the unknown parameters $\\theta$, the factor of $2$ cancels out."
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "62eff411",
+   "id": "87bea682",
    "metadata": {
     "editable": true
    },
@@ -292,19 +291,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "a70a3f6f",
+   "id": "592cb7cc",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "C(\\boldsymbol{\\beta})=\\frac{1}{n}\\left\\{\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)\\right\\},\n",
+    "C(\\boldsymbol{\\theta})=\\frac{1}{n}\\left\\{\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\right)\\right\\},\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "ca289772",
+   "id": "8ad2c594",
    "metadata": {
     "editable": true
    },
@@ -315,19 +314,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "a8312deb",
+   "id": "0d53b5e7",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "y_{i}=\\langle y_i \\rangle = \\beta_0x_{i,0}+\\beta_1x_{i,1}+\\beta_2x_{i,2}+\\dots+\\beta_{n-1}x_{i,n-1}+\\epsilon_i,\n",
+    "y_{i}=\\langle y_i \\rangle = \\theta_0x_{i,0}+\\theta_1x_{i,1}+\\theta_2x_{i,2}+\\dots+\\theta_{n-1}x_{i,n-1}+\\epsilon_i,\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "df1f3210",
+   "id": "a3cbbda7",
    "metadata": {
     "editable": true
    },
@@ -341,25 +340,25 @@
     "the standard deviation discussed earlier. In the discussion here we\n",
     "will treat $y_i$ as our exact value for the response variable.\n",
     "\n",
-    "In order to find the parameters $\\beta_i$ we will then minimize the spread of $C(\\boldsymbol{\\beta})$, that is we are going to solve the problem"
+    "In order to find the parameters $\\theta_i$ we will then minimize the spread of $C(\\boldsymbol{\\theta})$, that is we are going to solve the problem"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "20dd8ffe",
+   "id": "169c08ab",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "{\\displaystyle \\min_{\\boldsymbol{\\beta}\\in\n",
-    "{\\mathbb{R}}^{p}}}\\frac{1}{n}\\left\\{\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)\\right\\}.\n",
+    "{\\displaystyle \\min_{\\boldsymbol{\\theta}\\in\n",
+    "{\\mathbb{R}}^{p}}}\\frac{1}{n}\\left\\{\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\right)\\right\\}.\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "f5acd1e3",
+   "id": "710a21eb",
    "metadata": {
     "editable": true
    },
@@ -369,19 +368,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "f5f153bf",
+   "id": "f18b4513",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\frac{\\partial C(\\boldsymbol{\\beta})}{\\partial \\beta_j} = \\frac{\\partial }{\\partial \\beta_j}\\left[ \\frac{1}{n}\\sum_{i=0}^{n-1}\\left(y_i-\\beta_0x_{i,0}-\\beta_1x_{i,1}-\\beta_2x_{i,2}-\\dots-\\beta_{n-1}x_{i,n-1}\\right)^2\\right]=0,\n",
+    "\\frac{\\partial C(\\boldsymbol{\\theta})}{\\partial \\theta_j} = \\frac{\\partial }{\\partial \\theta_j}\\left[ \\frac{1}{n}\\sum_{i=0}^{n-1}\\left(y_i-\\theta_0x_{i,0}-\\theta_1x_{i,1}-\\theta_2x_{i,2}-\\dots-\\theta_{n-1}x_{i,n-1}\\right)^2\\right]=0,\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "bc60661c",
+   "id": "20b56c08",
    "metadata": {
     "editable": true
    },
@@ -391,19 +390,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "ad0c51c8",
+   "id": "7f6b7e59",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\frac{\\partial C(\\boldsymbol{\\beta})}{\\partial \\beta_j} = -\\frac{2}{n}\\left[ \\sum_{i=0}^{n-1}x_{ij}\\left(y_i-\\beta_0x_{i,0}-\\beta_1x_{i,1}-\\beta_2x_{i,2}-\\dots-\\beta_{n-1}x_{i,n-1}\\right)\\right]=0,\n",
+    "\\frac{\\partial C(\\boldsymbol{\\theta})}{\\partial \\theta_j} = -\\frac{2}{n}\\left[ \\sum_{i=0}^{n-1}x_{ij}\\left(y_i-\\theta_0x_{i,0}-\\theta_1x_{i,1}-\\theta_2x_{i,2}-\\dots-\\theta_{n-1}x_{i,n-1}\\right)\\right]=0,\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "299217eb",
+   "id": "36cdc402",
    "metadata": {
     "editable": true
    },
@@ -413,19 +412,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "37c4f3d2",
+   "id": "cb8648c4",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\frac{\\partial C(\\boldsymbol{\\beta})}{\\partial \\boldsymbol{\\beta}^T} = 0 = \\boldsymbol{X}^T\\left( \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right).\n",
+    "\\frac{\\partial C(\\boldsymbol{\\theta})}{\\partial \\boldsymbol{\\theta}^T} = 0 = \\boldsymbol{X}^T\\left( \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\right).\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "0aa3bf78",
+   "id": "9d60b674",
    "metadata": {
     "editable": true
    },
@@ -436,19 +435,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "c7cd2c51",
+   "id": "475a553b",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\frac{\\partial C(\\boldsymbol{\\beta})}{\\partial \\boldsymbol{\\beta}^T} = 0 = \\boldsymbol{X}^T\\left( \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right),\n",
+    "\\frac{\\partial C(\\boldsymbol{\\theta})}{\\partial \\boldsymbol{\\theta}^T} = 0 = \\boldsymbol{X}^T\\left( \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\right),\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "8188af0b",
+   "id": "69814fd7",
    "metadata": {
     "editable": true
    },
@@ -458,19 +457,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "bb587a70",
+   "id": "75c465dc",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\boldsymbol{X}^T\\boldsymbol{y} = \\boldsymbol{X}^T\\boldsymbol{X}\\boldsymbol{\\beta},\n",
+    "\\boldsymbol{X}^T\\boldsymbol{y} = \\boldsymbol{X}^T\\boldsymbol{X}\\boldsymbol{\\theta},\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "a0facf6c",
+   "id": "fb3f5818",
    "metadata": {
     "editable": true
    },
@@ -480,19 +479,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "a5128b98",
+   "id": "01c31fce",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\boldsymbol{\\beta} =\\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y}.\n",
+    "\\boldsymbol{\\theta} =\\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y}.\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "25915602",
+   "id": "869d8825",
    "metadata": {
     "editable": true
    },
@@ -512,7 +511,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "a47041a7",
+   "id": "86056e8c",
    "metadata": {
     "editable": true
    },
@@ -539,7 +538,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "94f82a14",
+   "id": "d36297d2",
    "metadata": {
     "editable": true
    },
@@ -551,7 +550,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "c10b4b3f",
+   "id": "42a61ab6",
    "metadata": {
     "editable": true
    },
@@ -563,7 +562,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "a2ff2fae",
+   "id": "6216fdfa",
    "metadata": {
     "editable": true
    },
@@ -579,7 +578,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "35369c98",
+   "id": "1d8f89cb",
    "metadata": {
     "editable": true
    },
@@ -597,7 +596,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "c3cb1b39",
+   "id": "4c3d1ea5",
    "metadata": {
     "editable": true
    },
@@ -609,7 +608,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "78fec5b2",
+   "id": "41d96182",
    "metadata": {
     "editable": true
    },
@@ -621,7 +620,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "c2cb8b77",
+   "id": "187e7d47",
    "metadata": {
     "editable": true
    },
@@ -632,7 +631,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "672e3ddc",
+   "id": "fe831c6b",
    "metadata": {
     "editable": true
    },
@@ -644,7 +643,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "3d652847",
+   "id": "5e9942fa",
    "metadata": {
     "editable": true
    },
@@ -654,7 +653,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "e4d1006a",
+   "id": "a1589dc4",
    "metadata": {
     "editable": true
    },
@@ -666,7 +665,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "27e8e9da",
+   "id": "f4980fc7",
    "metadata": {
     "editable": true
    },
@@ -680,7 +679,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "f1b4e3e2",
+   "id": "5e5803ca",
    "metadata": {
     "editable": true
    },
@@ -692,7 +691,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "d247b10c",
+   "id": "8ff2d8a0",
    "metadata": {
     "editable": true
    },
@@ -704,7 +703,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "9d9b5cae",
+   "id": "a14f7d1e",
    "metadata": {
     "editable": true
    },
@@ -716,7 +715,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "28bc97b3",
+   "id": "8d0c272b",
    "metadata": {
     "editable": true
    },
@@ -726,7 +725,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "9bd668ca",
+   "id": "9bfc54a6",
    "metadata": {
     "editable": true
    },
@@ -738,7 +737,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "1f072069",
+   "id": "f68602df",
    "metadata": {
     "editable": true
    },
@@ -750,7 +749,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "468e0aaf",
+   "id": "f3a9b46b",
    "metadata": {
     "editable": true
    },
@@ -762,7 +761,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "df18a7bc",
+   "id": "8b9c9796",
    "metadata": {
     "editable": true
    },
@@ -776,7 +775,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "7e778a20",
+   "id": "e55c0131",
    "metadata": {
     "editable": true
    },
@@ -788,7 +787,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "7b09debd",
+   "id": "0213690c",
    "metadata": {
     "editable": true
    },
@@ -800,7 +799,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "bf4df408",
+   "id": "4bebb2ab",
    "metadata": {
     "editable": true
    },
@@ -812,7 +811,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "9527ed06",
+   "id": "f90da960",
    "metadata": {
     "editable": true
    },
@@ -822,7 +821,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "4b2b0605",
+   "id": "8178ffe4",
    "metadata": {
     "editable": true
    },
@@ -834,7 +833,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "8a355b26",
+   "id": "d937141e",
    "metadata": {
     "editable": true
    },
@@ -844,7 +843,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "ac595cbf",
+   "id": "a5a8213f",
    "metadata": {
     "editable": true
    },
@@ -856,7 +855,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "823f52d0",
+   "id": "3bdb5bee",
    "metadata": {
     "editable": true
    },
@@ -866,7 +865,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "a90148b7",
+   "id": "4a9ff5dc",
    "metadata": {
     "editable": true
    },
@@ -878,7 +877,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "d923e6f1",
+   "id": "97098140",
    "metadata": {
     "editable": true
    },
@@ -890,7 +889,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "1106be29",
+   "id": "76008e78",
    "metadata": {
     "editable": true
    },
@@ -902,7 +901,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "5ed90732",
+   "id": "7e07b9bf",
    "metadata": {
     "editable": true
    },
@@ -917,7 +916,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "d516517b",
+   "id": "a691ed11",
    "metadata": {
     "editable": true
    },
@@ -929,7 +928,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "c1379242",
+   "id": "73cfbba0",
    "metadata": {
     "editable": true
    },
@@ -939,7 +938,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "3724106d",
+   "id": "d78a9df5",
    "metadata": {
     "editable": true
    },
@@ -951,7 +950,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "e085c2ec",
+   "id": "9e1e3a87",
    "metadata": {
     "editable": true
    },
@@ -961,7 +960,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "01fa0478",
+   "id": "aea5a5e4",
    "metadata": {
     "editable": true
    },
@@ -973,7 +972,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "8c75a8cd",
+   "id": "89e6fde8",
    "metadata": {
     "editable": true
    },
@@ -983,7 +982,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "e3e275e6",
+   "id": "6c0f4449",
    "metadata": {
     "editable": true
    },
@@ -995,7 +994,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "ab39269e",
+   "id": "a390ac6e",
    "metadata": {
     "editable": true
    },
@@ -1007,19 +1006,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "ce0b7d4b",
+   "id": "59d6ab1c",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "C(\\boldsymbol{\\beta})=\\frac{1}{n}\\sum_{i=0}^{n-1}\\left(y_i-\\tilde{y}_i\\right)^2=\\frac{1}{n}\\left\\{\\left(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}}\\right)\\right\\},\n",
+    "C(\\boldsymbol{\\theta})=\\frac{1}{n}\\sum_{i=0}^{n-1}\\left(y_i-\\tilde{y}_i\\right)^2=\\frac{1}{n}\\left\\{\\left(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}}\\right)\\right\\},\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "31fbfca8",
+   "id": "8c99ee21",
    "metadata": {
     "editable": true
    },
@@ -1029,66 +1028,66 @@
   },
   {
    "cell_type": "markdown",
-   "id": "f3506f4e",
+   "id": "7ea454e1",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "C(\\boldsymbol{\\beta})=\\frac{1}{n}\\left\\{\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)\\right\\}.\n",
+    "C(\\boldsymbol{\\theta})=\\frac{1}{n}\\left\\{\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\right)\\right\\}.\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "6f709ae5",
+   "id": "5a555af4",
    "metadata": {
     "editable": true
    },
    "source": [
-    "We note that the design matrix $\\boldsymbol{X}$ does not depend on the unknown parameters defined by the vector $\\boldsymbol{\\beta}$.\n",
-    "We are now interested in minimizing the cost function with respect to the unknown parameters $\\boldsymbol{\\beta}$.\n",
+    "We note that the design matrix $\\boldsymbol{X}$ does not depend on the unknown parameters defined by the vector $\\boldsymbol{\\theta}$.\n",
+    "We are now interested in minimizing the cost function with respect to the unknown parameters $\\boldsymbol{\\theta}$.\n",
     "\n",
     "The mean squared error is a scalar and if we use the results from example three above, we can define a new vector"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "02fba45f",
+   "id": "a39c02c1",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\boldsymbol{w}=\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta},\n",
+    "\\boldsymbol{w}=\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta},\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "af072826",
+   "id": "a9804433",
    "metadata": {
     "editable": true
    },
    "source": [
-    "which depends on $\\boldsymbol{\\beta}$. We rewrite the cost function as"
+    "which depends on $\\boldsymbol{\\theta}$. We rewrite the cost function as"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "35947153",
+   "id": "8be8de93",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "C(\\boldsymbol{\\beta})=\\frac{1}{n}\\boldsymbol{w}^T\\boldsymbol{w},\n",
+    "C(\\boldsymbol{\\theta})=\\frac{1}{n}\\boldsymbol{w}^T\\boldsymbol{w},\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "84e2032b",
+   "id": "4139dd13",
    "metadata": {
     "editable": true
    },
@@ -1098,19 +1097,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "46895a0e",
+   "id": "6f1c778a",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\frac{\\partial C(\\boldsymbol{\\beta})}{\\partial \\boldsymbol{\\beta}}=\\frac{2}{n}\\boldsymbol{w}^T\\frac{\\partial \\boldsymbol{w}}{\\partial \\boldsymbol{\\beta}},\n",
+    "\\frac{\\partial C(\\boldsymbol{\\theta})}{\\partial \\boldsymbol{\\theta}}=\\frac{2}{n}\\boldsymbol{w}^T\\frac{\\partial \\boldsymbol{w}}{\\partial \\boldsymbol{\\theta}},\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "b2ffaf06",
+   "id": "03ae847f",
    "metadata": {
     "editable": true
    },
@@ -1120,19 +1119,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "1f591640",
+   "id": "054f8ba7",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\frac{\\partial \\boldsymbol{w}}{\\partial \\boldsymbol{\\beta}}=-\\boldsymbol{X},\n",
+    "\\frac{\\partial \\boldsymbol{w}}{\\partial \\boldsymbol{\\theta}}=-\\boldsymbol{X},\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "0322fd59",
+   "id": "61c62efa",
    "metadata": {
     "editable": true
    },
@@ -1142,19 +1141,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "1e03255f",
+   "id": "ac84f4fd",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\frac{\\partial C(\\boldsymbol{\\beta})}{\\partial \\boldsymbol{\\beta}}=-\\frac{2}{n}\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)^T\\boldsymbol{X},\n",
+    "\\frac{\\partial C(\\boldsymbol{\\theta})}{\\partial \\boldsymbol{\\theta}}=-\\frac{2}{n}\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\right)^T\\boldsymbol{X},\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "0a1403f8",
+   "id": "b2f0a3d0",
    "metadata": {
     "editable": true
    },
@@ -1164,19 +1163,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "41a42e64",
+   "id": "9c6460f2",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\frac{\\partial C(\\boldsymbol{\\beta})}{\\partial \\boldsymbol{\\beta}^T}=-\\frac{2}{n}\\boldsymbol{X}^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right).\n",
+    "\\frac{\\partial C(\\boldsymbol{\\theta})}{\\partial \\boldsymbol{\\theta}^T}=-\\frac{2}{n}\\boldsymbol{X}^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\right).\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "e68e4f65",
+   "id": "0c4aebc4",
    "metadata": {
     "editable": true
    },
@@ -1188,7 +1187,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "9f4ecb7d",
+   "id": "67aef36e",
    "metadata": {
     "editable": true
    },
@@ -1200,7 +1199,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "d050fc18",
+   "id": "30c88be7",
    "metadata": {
     "editable": true
    },
@@ -1212,7 +1211,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "67837a5c",
+   "id": "fb84fcda",
    "metadata": {
     "editable": true
    },
@@ -1224,7 +1223,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "562ad5bd",
+   "id": "ebbeeb3b",
    "metadata": {
     "editable": true
    },
@@ -1236,7 +1235,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "bf5751b0",
+   "id": "fb88afbf",
    "metadata": {
     "editable": true
    },
@@ -1245,26 +1244,26 @@
     "\n",
     "A very important matrix we will meet again and again in machine\n",
     "learning is the Hessian.  It is given by the second derivative of the\n",
-    "cost function with respect to the parameters $\\boldsymbol{\\beta}$. Using the above\n",
+    "cost function with respect to the parameters $\\boldsymbol{\\theta}$. Using the above\n",
     "expression for derivatives of vectors and matrices, we find that the\n",
     "second derivative of the mean squared error as cost function is,"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "6f1a224c",
+   "id": "89512329",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\frac{\\partial}{\\partial \\boldsymbol{\\beta}}\\frac{\\partial C(\\boldsymbol{\\beta})}{\\partial \\boldsymbol{\\beta}^T} =\\frac{\\partial}{\\partial \\boldsymbol{\\beta}}\\left[-\\frac{2}{n}\\boldsymbol{X}^T\\left( \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)\\right]=\\frac{2}{n}\\boldsymbol{X}^T\\boldsymbol{X}.\n",
+    "\\frac{\\partial}{\\partial \\boldsymbol{\\theta}}\\frac{\\partial C(\\boldsymbol{\\theta})}{\\partial \\boldsymbol{\\theta}^T} =\\frac{\\partial}{\\partial \\boldsymbol{\\theta}}\\left[-\\frac{2}{n}\\boldsymbol{X}^T\\left( \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\right)\\right]=\\frac{2}{n}\\boldsymbol{X}^T\\boldsymbol{X}.\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "4c9387ca",
+   "id": "a651d788",
    "metadata": {
     "editable": true
    },
@@ -1274,7 +1273,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "e55cd91c",
+   "id": "dca9d68f",
    "metadata": {
     "editable": true
    },
@@ -1286,14 +1285,14 @@
   },
   {
    "cell_type": "markdown",
-   "id": "5834e282",
+   "id": "2865ced8",
    "metadata": {
     "editable": true
    },
    "source": [
     "For ordinary least squares, it is inversely proportional (derivation\n",
     "next week) with the variance of the optimal parameters\n",
-    "$\\hat{\\boldsymbol{\\beta}}$. Furthermore, we will see later this week that it is\n",
+    "$\\hat{\\boldsymbol{\\theta}}$. Furthermore, we will see later this week that it is\n",
     "(aside the factor $1/n$) equal to the covariance matrix. It plays also a very\n",
     "important role in optmization algorithms and Principal Component\n",
     "Analysis as a way to reduce the dimensionality of a machine learning/data analysis\n",
@@ -1304,7 +1303,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "d79e83c1",
+   "id": "1a7a4045",
    "metadata": {
     "editable": true
    },
@@ -1316,19 +1315,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "435ad3ea",
+   "id": "a483362e",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\boldsymbol{\\epsilon} = \\boldsymbol{y}-\\boldsymbol{\\tilde{y}} = \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta},\n",
+    "\\boldsymbol{\\epsilon} = \\boldsymbol{y}-\\boldsymbol{\\tilde{y}} = \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta},\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "3d0c550b",
+   "id": "dcf896a3",
    "metadata": {
     "editable": true
    },
@@ -1338,19 +1337,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "1317ebae",
+   "id": "13c80a7c",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\boldsymbol{X}^T\\left( \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)= 0,\n",
+    "\\boldsymbol{X}^T\\left( \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\right)= 0,\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "c8c5a477",
+   "id": "582225c6",
    "metadata": {
     "editable": true
    },
@@ -1360,29 +1359,29 @@
   },
   {
    "cell_type": "markdown",
-   "id": "93d276b1",
+   "id": "90476b18",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\boldsymbol{X}^T\\boldsymbol{\\epsilon}=\\boldsymbol{X}^T\\left( \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)= 0,\n",
+    "\\boldsymbol{X}^T\\boldsymbol{\\epsilon}=\\boldsymbol{X}^T\\left( \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\right)= 0,\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "b292af90",
+   "id": "267ae1ff",
    "metadata": {
     "editable": true
    },
    "source": [
-    "meaning that the solution for $\\boldsymbol{\\beta}$ is the one which minimizes the residuals."
+    "meaning that the solution for $\\boldsymbol{\\theta}$ is the one which minimizes the residuals."
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "e8b71fe3",
+   "id": "8c9cb8c5",
    "metadata": {
     "editable": true
    },
@@ -1396,43 +1395,43 @@
   },
   {
    "cell_type": "markdown",
-   "id": "b5b88839",
+   "id": "acbe4114",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\tilde{y}_i = \\beta_0+\\beta_1x_i+\\beta_2x_i^2+\\beta_3x_i^3+\\beta_4x_i^4.\n",
+    "\\tilde{y}_i = \\theta_0+\\theta_1x_i+\\theta_2x_i^2+\\theta_3x_i^3+\\theta_4x_i^4.\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "4b65ef7c",
+   "id": "ba945f21",
    "metadata": {
     "editable": true
    },
    "source": [
-    "we have five predictors/features. The first is the intercept $\\beta_0$. The other terms are $\\beta_i$ with $i=1,2,3,4$. Furthermore we have $n$ entries for each predictor. It means that our design matrix is an \n",
+    "we have five predictors/features. The first is the intercept $\\theta_0$. The other terms are $\\theta_i$ with $i=1,2,3,4$. Furthermore we have $n$ entries for each predictor. It means that our design matrix is an \n",
     "$n\\times p$ matrix $\\boldsymbol{X}$."
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "36fa0c5d",
+   "id": "dc442b90",
    "metadata": {
     "editable": true
    },
    "source": [
     "## Own code for Ordinary Least Squares\n",
     "\n",
-    "It is rather straightforward to implement the matrix inversion and obtain the parameters $\\boldsymbol{\\beta}$. After having defined the matrix $\\boldsymbol{X}$ and the outputs $\\boldsymbol{y}$ we have"
+    "It is rather straightforward to implement the matrix inversion and obtain the parameters $\\boldsymbol{\\theta}$. After having defined the matrix $\\boldsymbol{X}$ and the outputs $\\boldsymbol{y}$ we have"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 1,
-   "id": "5bfc7faa",
+   "id": "883026f0",
    "metadata": {
     "collapsed": false,
     "editable": true
@@ -1459,7 +1458,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "07764fc2",
+   "id": "826466d3",
    "metadata": {
     "editable": true
    },
@@ -1470,7 +1469,7 @@
   {
    "cell_type": "code",
    "execution_count": 2,
-   "id": "f299da23",
+   "id": "decea3b4",
    "metadata": {
     "collapsed": false,
     "editable": true
@@ -1483,7 +1482,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "839633de",
+   "id": "b52ed870",
    "metadata": {
     "editable": true
    },
@@ -1497,7 +1496,7 @@
   {
    "cell_type": "code",
    "execution_count": 3,
-   "id": "abfc6c01",
+   "id": "ee4d5722",
    "metadata": {
     "collapsed": false,
     "editable": true
@@ -1510,7 +1509,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "aff33949",
+   "id": "01892aaa",
    "metadata": {
     "editable": true
    },
@@ -1521,7 +1520,7 @@
   {
    "cell_type": "code",
    "execution_count": 4,
-   "id": "67e83121",
+   "id": "d6ce54e2",
    "metadata": {
     "collapsed": false,
     "editable": true
@@ -1533,7 +1532,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "14a294e9",
+   "id": "91ed5956",
    "metadata": {
     "editable": true
    },
@@ -1544,7 +1543,7 @@
   {
    "cell_type": "code",
    "execution_count": 5,
-   "id": "8d80a11c",
+   "id": "7ffd63cb",
    "metadata": {
     "collapsed": false,
     "editable": true
@@ -1560,7 +1559,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "fd9c909e",
+   "id": "737370ed",
    "metadata": {
     "editable": true
    },
@@ -1571,7 +1570,7 @@
   {
    "cell_type": "code",
    "execution_count": 6,
-   "id": "fcd82857",
+   "id": "f1bbe22d",
    "metadata": {
     "collapsed": false,
     "editable": true
@@ -1585,7 +1584,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "a91adf8f",
+   "id": "8e3a976b",
    "metadata": {
     "editable": true
    },
@@ -1606,7 +1605,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "24f5afb4",
+   "id": "8c2dee8c",
    "metadata": {
     "editable": true
    },
@@ -1617,7 +1616,7 @@
   {
    "cell_type": "code",
    "execution_count": 7,
-   "id": "093bd1f1",
+   "id": "89d0750d",
    "metadata": {
     "collapsed": false,
     "editable": true
@@ -1670,7 +1669,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "08a5d650",
+   "id": "12c44026",
    "metadata": {
     "editable": true
    },
@@ -1681,7 +1680,7 @@
   {
    "cell_type": "code",
    "execution_count": 8,
-   "id": "26c93a00",
+   "id": "6daa8ad9",
    "metadata": {
     "collapsed": false,
     "editable": true
@@ -1706,7 +1705,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "0b3c6835",
+   "id": "9b4cbdbc",
    "metadata": {
     "editable": true
    },
@@ -1718,7 +1717,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "7eae01b1",
+   "id": "7f685bde",
    "metadata": {
     "editable": true
    },
@@ -1747,7 +1746,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "f4576ab0",
+   "id": "a8e92a0b",
    "metadata": {
     "editable": true
    },
@@ -1772,7 +1771,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "04b757cc",
+   "id": "b0ac1e21",
    "metadata": {
     "editable": true
    },
@@ -1792,7 +1791,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "e96e0ec7",
+   "id": "3c8d9613",
    "metadata": {
     "editable": true
    },
@@ -1819,7 +1818,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "b4e7944d",
+   "id": "1d817e20",
    "metadata": {
     "editable": true
    },
@@ -1832,7 +1831,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "385fbd02",
+   "id": "20673f07",
    "metadata": {
     "editable": true
    },
@@ -1844,7 +1843,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "65176139",
+   "id": "4a36afc7",
    "metadata": {
     "editable": true
    },
@@ -1855,7 +1854,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "bc2bda95",
+   "id": "4db40306",
    "metadata": {
     "editable": true
    },
@@ -1871,7 +1870,7 @@
   {
    "cell_type": "code",
    "execution_count": 9,
-   "id": "f741746f",
+   "id": "ac782b4e",
    "metadata": {
     "collapsed": false,
     "editable": true
@@ -1905,7 +1904,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "91b66d01",
+   "id": "d60792bf",
    "metadata": {
     "editable": true
    },
@@ -1915,7 +1914,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "376037d2",
+   "id": "6dbeeee3",
    "metadata": {
     "editable": true
    },
@@ -1930,7 +1929,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "3b6fa136",
+   "id": "f1bd9e06",
    "metadata": {
     "editable": true
    },
@@ -1942,7 +1941,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "f1087613",
+   "id": "c4e9c9cc",
    "metadata": {
     "editable": true
    },
@@ -1952,7 +1951,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "be2cd7be",
+   "id": "c682dfb2",
    "metadata": {
     "editable": true
    },
@@ -1968,7 +1967,7 @@
   {
    "cell_type": "code",
    "execution_count": 10,
-   "id": "0ac0921a",
+   "id": "f95e4477",
    "metadata": {
     "collapsed": false,
     "editable": true
@@ -1985,7 +1984,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "700d2c70",
+   "id": "3eaa6d2e",
    "metadata": {
     "editable": true
    },
@@ -1998,7 +1997,7 @@
   {
    "cell_type": "code",
    "execution_count": 11,
-   "id": "3556cc72",
+   "id": "fe340fe4",
    "metadata": {
     "collapsed": false,
     "editable": true
@@ -2045,7 +2044,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "88222231",
+   "id": "e2f14a01",
    "metadata": {
     "editable": true
    },
@@ -2054,48 +2053,48 @@
     "\n",
     "What is presented here is a mathematical analysis of various regression algorithms (ordinary least  squares, Ridge and Lasso Regression). The analysis is based on an important algorithm in linear algebra, the so-called Singular Value Decomposition (SVD). \n",
     "\n",
-    "We have shown that in ordinary least squares the optimal parameters $\\beta$ are given by"
+    "We have shown that in ordinary least squares the optimal parameters $\\theta$ are given by"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "9fa5b4c6",
+   "id": "ad741d5a",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\hat{\\boldsymbol{\\beta}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y}.\n",
+    "\\hat{\\boldsymbol{\\theta}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y}.\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "81bb7765",
+   "id": "05b080a8",
    "metadata": {
     "editable": true
    },
    "source": [
-    "The **hat** over $\\boldsymbol{\\beta}$ means we have the optimal parameters after minimization of the cost function.\n",
+    "The **hat** over $\\boldsymbol{\\theta}$ means we have the optimal parameters after minimization of the cost function.\n",
     "\n",
     "This means that our best model is defined as"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "e6d5287b",
+   "id": "5c64cd00",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\tilde{\\boldsymbol{y}}=\\boldsymbol{X}\\hat{\\boldsymbol{\\beta}} = \\boldsymbol{X}\\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y}.\n",
+    "\\tilde{\\boldsymbol{y}}=\\boldsymbol{X}\\hat{\\boldsymbol{\\theta}} = \\boldsymbol{X}\\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y}.\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "c95f3611",
+   "id": "66d8648a",
    "metadata": {
     "editable": true
    },
@@ -2105,7 +2104,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "b688fc2e",
+   "id": "47178854",
    "metadata": {
     "editable": true
    },
@@ -2117,7 +2116,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "3109192b",
+   "id": "9d89dcc7",
    "metadata": {
     "editable": true
    },
@@ -2127,19 +2126,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "833ca3e1",
+   "id": "d037dda7",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\tilde{\\boldsymbol{y}}=\\boldsymbol{X}\\hat{\\boldsymbol{\\beta}} = \\boldsymbol{A}\\boldsymbol{y}.\n",
+    "\\tilde{\\boldsymbol{y}}=\\boldsymbol{X}\\hat{\\boldsymbol{\\theta}} = \\boldsymbol{A}\\boldsymbol{y}.\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "f2f8408c",
+   "id": "4d559b4d",
    "metadata": {
     "editable": true
    },
@@ -2150,7 +2149,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "b93b6e5d",
+   "id": "f8eec117",
    "metadata": {
     "editable": true
    },
@@ -2162,7 +2161,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "71a3aeb6",
+   "id": "ecc2fed3",
    "metadata": {
     "editable": true
    },
@@ -2174,7 +2173,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "cd016113",
+   "id": "8d3ae448",
    "metadata": {
     "editable": true
    },
@@ -2184,7 +2183,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "510e85fb",
+   "id": "0d7267fa",
    "metadata": {
     "editable": true
    },
@@ -2196,7 +2195,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "b00cee57",
+   "id": "06e5716f",
    "metadata": {
     "editable": true
    },
@@ -2208,7 +2207,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "9ee31587",
+   "id": "f8f001a6",
    "metadata": {
     "editable": true
    },
@@ -2218,7 +2217,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "deca37bb",
+   "id": "761f6c51",
    "metadata": {
     "editable": true
    },
@@ -2230,7 +2229,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "836d9d6c",
+   "id": "468bc321",
    "metadata": {
     "editable": true
    },
@@ -2240,7 +2239,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "eee14cd2",
+   "id": "eb61e7ef",
    "metadata": {
     "editable": true
    },
@@ -2252,7 +2251,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "40906c45",
+   "id": "be3586ce",
    "metadata": {
     "editable": true
    },
@@ -2262,7 +2261,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "09e6f2ed",
+   "id": "f79b7c51",
    "metadata": {
     "editable": true
    },
@@ -2302,7 +2301,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "ffd73218",
+   "id": "ea84dc35",
    "metadata": {
     "editable": true
    },
@@ -2319,7 +2318,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "93e66912",
+   "id": "0ad4b3f9",
    "metadata": {
     "editable": true
    },
@@ -2342,7 +2341,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "600d07ab",
+   "id": "671100e8",
    "metadata": {
     "editable": true
    },
@@ -2359,7 +2358,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "46e320a9",
+   "id": "9d07ce4a",
    "metadata": {
     "editable": true
    },
@@ -2378,7 +2377,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "b23338e4",
+   "id": "a26b13de",
    "metadata": {
     "editable": true
    },
@@ -2389,7 +2388,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "bc566714",
+   "id": "2352aaa2",
    "metadata": {
     "editable": true
    },
@@ -2401,7 +2400,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "ea1b8269",
+   "id": "5c201fe7",
    "metadata": {
     "editable": true
    },
@@ -2411,7 +2410,7 @@
     "\n",
     "$$\n",
     "\\begin{equation}\n",
-    "\\boldsymbol{\\beta}  =  (\\boldsymbol{X}^{T} \\boldsymbol{X})^{-1} \\boldsymbol{X}^{T} \\boldsymbol{y},\n",
+    "\\boldsymbol{\\theta}  =  (\\boldsymbol{X}^{T} \\boldsymbol{X})^{-1} \\boldsymbol{X}^{T} \\boldsymbol{y},\n",
     "\\label{_auto1} \\tag{1}\n",
     "\\end{equation}\n",
     "$$"
@@ -2419,23 +2418,23 @@
   },
   {
    "cell_type": "markdown",
-   "id": "9aee9220",
+   "id": "a711f9aa",
    "metadata": {
     "editable": true
    },
    "source": [
     "has linearly dependent column vectors, we will not be able to compute the inverse\n",
-    "of $\\boldsymbol{X}^T\\boldsymbol{X}$ and we cannot find the parameters (estimators) $\\beta_i$. \n",
+    "of $\\boldsymbol{X}^T\\boldsymbol{X}$ and we cannot find the parameters (estimators) $\\theta_i$. \n",
     "The estimators are only well-defined if $(\\boldsymbol{X}^{T}\\boldsymbol{X})^{-1}$ exits. \n",
     "This is more likely to happen when the matrix $\\boldsymbol{X}$ is high-dimensional. In this case it is likely to encounter a situation where \n",
-    "the regression parameters $\\beta_i$ cannot be estimated.\n",
+    "the regression parameters $\\theta_i$ cannot be estimated.\n",
     "\n",
     "A cheap  *ad hoc* approach is  simply to add a small diagonal component to the matrix to invert, that is we change"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "61e7fbda",
+   "id": "816ee812",
    "metadata": {
     "editable": true
    },
@@ -2447,7 +2446,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "afcc930d",
+   "id": "ca4f2d5d",
    "metadata": {
     "editable": true
    },
@@ -2457,7 +2456,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "6ed4a916",
+   "id": "11b18875",
    "metadata": {
     "editable": true
    },
@@ -2472,7 +2471,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "20949d31",
+   "id": "a6cebe96",
    "metadata": {
     "editable": true
    },
@@ -2484,7 +2483,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "1ad7169f",
+   "id": "4056c51d",
    "metadata": {
     "editable": true
    },
@@ -2494,7 +2493,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "4d8a46e3",
+   "id": "8c6fc251",
    "metadata": {
     "editable": true
    },
@@ -2506,7 +2505,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "79a96264",
+   "id": "85d151ae",
    "metadata": {
     "editable": true
    },
@@ -2516,7 +2515,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "0d30c476",
+   "id": "a7ed48ac",
    "metadata": {
     "editable": true
    },
@@ -2528,7 +2527,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "d8320347",
+   "id": "d61100de",
    "metadata": {
     "editable": true
    },
@@ -2540,7 +2539,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "54e7ef4a",
+   "id": "ae931201",
    "metadata": {
     "editable": true
    },
@@ -2555,7 +2554,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "ea96bab9",
+   "id": "a1a55970",
    "metadata": {
     "editable": true
    },
@@ -2566,7 +2565,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "8f610591",
+   "id": "7aff14ea",
    "metadata": {
     "editable": true
    },
@@ -2586,7 +2585,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "3cbcadcc",
+   "id": "c5bcd441",
    "metadata": {
     "editable": true
    },
@@ -2598,7 +2597,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "3d6fa2f6",
+   "id": "ed33684a",
    "metadata": {
     "editable": true
    },
@@ -2608,7 +2607,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "03adee16",
+   "id": "1d6ca687",
    "metadata": {
     "editable": true
    },
@@ -2620,7 +2619,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "e2f52707",
+   "id": "c0535cd8",
    "metadata": {
     "editable": true
    },
@@ -2649,7 +2648,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "143557e4",
+   "id": "c6c08bf2",
    "metadata": {
     "editable": true
    },
@@ -2676,7 +2675,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "8c5068c0",
+   "id": "487a7ae7",
    "metadata": {
     "editable": true
    },
@@ -2687,7 +2686,7 @@
   {
    "cell_type": "code",
    "execution_count": 12,
-   "id": "090ea41c",
+   "id": "3ff87df9",
    "metadata": {
     "collapsed": false,
     "editable": true
@@ -2727,7 +2726,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "7d055231",
+   "id": "600f74f5",
    "metadata": {
     "editable": true
    },
@@ -2744,7 +2743,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "5d1a15fe",
+   "id": "da923c99",
    "metadata": {
     "editable": true
    },
@@ -2767,7 +2766,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "ee3105fb",
+   "id": "989e9de3",
    "metadata": {
     "editable": true
    },
@@ -2781,7 +2780,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "b62baed5",
+   "id": "b010904b",
    "metadata": {
     "editable": true
    },
@@ -2800,7 +2799,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "1c4c8788",
+   "id": "1385bdc7",
    "metadata": {
     "editable": true
    },
@@ -2810,7 +2809,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "a6c47646",
+   "id": "1824f1de",
    "metadata": {
     "editable": true
    },
@@ -2822,7 +2821,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "d5903c9f",
+   "id": "50fb0869",
    "metadata": {
     "editable": true
    },
@@ -2836,7 +2835,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "64e17d36",
+   "id": "242e0186",
    "metadata": {
     "editable": true
    },
@@ -2848,7 +2847,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "572c9d3f",
+   "id": "b3d8ff35",
    "metadata": {
     "editable": true
    },
@@ -2858,7 +2857,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "3f1a2cb9",
+   "id": "a8799cf8",
    "metadata": {
     "editable": true
    },
@@ -2870,7 +2869,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "41708552",
+   "id": "69ecbf25",
    "metadata": {
     "editable": true
    },
@@ -2887,7 +2886,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "97003366",
+   "id": "accaad01",
    "metadata": {
     "editable": true
    },
@@ -2897,7 +2896,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "3bb5d954",
+   "id": "7c6d6458",
    "metadata": {
     "editable": true
    },
@@ -2913,7 +2912,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "0c60d5d7",
+   "id": "92943a13",
    "metadata": {
     "editable": true
    },
@@ -2923,7 +2922,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "bcb087ce",
+   "id": "27a56b68",
    "metadata": {
     "editable": true
    },
@@ -2939,7 +2938,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "b718bf55",
+   "id": "7fab099a",
    "metadata": {
     "editable": true
    },
@@ -2949,7 +2948,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "3d4eef3e",
+   "id": "86ef772d",
    "metadata": {
     "editable": true
    },
@@ -2965,7 +2964,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "9a61fbcc",
+   "id": "a9fb58a3",
    "metadata": {
     "editable": true
    },
@@ -2975,7 +2974,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "bc19b5f2",
+   "id": "218d36d5",
    "metadata": {
     "editable": true
    },
@@ -2992,7 +2991,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "4449fbb0",
+   "id": "3c358960",
    "metadata": {
     "editable": true
    },
@@ -3004,7 +3003,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "31bcf6bb",
+   "id": "384a8da4",
    "metadata": {
     "editable": true
    },
@@ -3016,7 +3015,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "844ac1e6",
+   "id": "506802df",
    "metadata": {
     "editable": true
    },
@@ -3028,7 +3027,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "22f3a41d",
+   "id": "d160546b",
    "metadata": {
     "editable": true
    },
@@ -3038,7 +3037,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "15d8fff5",
+   "id": "22fa787b",
    "metadata": {
     "editable": true
    },
@@ -3050,7 +3049,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "430bf7ee",
+   "id": "4f9cc0fa",
    "metadata": {
     "editable": true
    },
@@ -3062,7 +3061,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "720d4fb8",
+   "id": "5228fab9",
    "metadata": {
     "editable": true
    },
@@ -3074,7 +3073,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "8deee9d7",
+   "id": "803c129e",
    "metadata": {
     "editable": true
    },
@@ -3084,7 +3083,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "897ea593",
+   "id": "a0229d06",
    "metadata": {
     "editable": true
    },
@@ -3096,7 +3095,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "5321f718",
+   "id": "927c0677",
    "metadata": {
     "editable": true
    },
@@ -3106,7 +3105,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "ba34da1a",
+   "id": "9b1faff4",
    "metadata": {
     "editable": true
    },
@@ -3118,7 +3117,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "0e8a413f",
+   "id": "2a9ba785",
    "metadata": {
     "editable": true
    },
@@ -3134,7 +3133,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "dd064267",
+   "id": "13dd8452",
    "metadata": {
     "editable": true
    },
@@ -3146,7 +3145,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "3a1bc23a",
+   "id": "db18ae22",
    "metadata": {
     "editable": true
    },
@@ -3158,7 +3157,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "3bc120c3",
+   "id": "cc002ef4",
    "metadata": {
     "editable": true
    },
@@ -3168,7 +3167,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "fa44e331",
+   "id": "7808b2cd",
    "metadata": {
     "editable": true
    },
@@ -3180,7 +3179,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "2f7d8c59",
+   "id": "f6a8e671",
    "metadata": {
     "editable": true
    },
@@ -3191,7 +3190,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "e5c786e1",
+   "id": "e6298500",
    "metadata": {
     "editable": true
    },
@@ -3203,7 +3202,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "d4c7d4c6",
+   "id": "1a70e57a",
    "metadata": {
     "editable": true
    },
@@ -3213,7 +3212,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "c8bf0455",
+   "id": "f2091227",
    "metadata": {
     "editable": true
    },
@@ -3225,7 +3224,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "b8a2bdb3",
+   "id": "029a7fce",
    "metadata": {
     "editable": true
    },
@@ -3235,7 +3234,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "20e30049",
+   "id": "62dd4376",
    "metadata": {
     "editable": true
    },
@@ -3247,7 +3246,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "801ec139",
+   "id": "00865f17",
    "metadata": {
     "editable": true
    },
@@ -3258,7 +3257,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "66a71bf6",
+   "id": "64bd2f73",
    "metadata": {
     "editable": true
    },
@@ -3270,7 +3269,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "975d23d9",
+   "id": "ae49067e",
    "metadata": {
     "editable": true
    },
@@ -3288,7 +3287,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "93c0159c",
+   "id": "9ea73c78",
    "metadata": {
     "editable": true
    },
@@ -3304,19 +3303,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "29add97b",
+   "id": "6dcf8fff",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\frac{\\partial^2 C(\\boldsymbol{\\beta})}{\\partial \\boldsymbol{\\beta}\\partial \\boldsymbol{\\beta}^T} =\\frac{2}{n}\\boldsymbol{X}^T\\boldsymbol{X}.\n",
+    "\\frac{\\partial^2 C(\\boldsymbol{\\theta})}{\\partial \\boldsymbol{\\theta}\\partial \\boldsymbol{\\theta}^T} =\\frac{2}{n}\\boldsymbol{X}^T\\boldsymbol{X}.\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "b17c198b",
+   "id": "8f12f6ca",
    "metadata": {
     "editable": true
    },
@@ -3328,7 +3327,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "ea1d3d0b",
+   "id": "03362376",
    "metadata": {
     "editable": true
    },
@@ -3340,7 +3339,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "a169ccb3",
+   "id": "76d4b169",
    "metadata": {
     "editable": true
    },
@@ -3353,7 +3352,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "c90bce62",
+   "id": "6775daac",
    "metadata": {
     "editable": true
    },
@@ -3369,7 +3368,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "3b26baba",
+   "id": "dc1392b1",
    "metadata": {
     "editable": true
    },
@@ -3383,7 +3382,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "8331409e",
+   "id": "7faf1d79",
    "metadata": {
     "editable": true
    },
@@ -3393,7 +3392,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "4d391ba3",
+   "id": "4c148d48",
    "metadata": {
     "editable": true
    },
@@ -3405,7 +3404,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "46947320",
+   "id": "d2549195",
    "metadata": {
     "editable": true
    },
@@ -3415,7 +3414,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "acf0ace2",
+   "id": "4d7f14be",
    "metadata": {
     "editable": true
    },
@@ -3427,7 +3426,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "02563a13",
+   "id": "3d795b82",
    "metadata": {
     "editable": true
    },
@@ -3437,7 +3436,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "349de4e6",
+   "id": "033e1b21",
    "metadata": {
     "editable": true
    },
@@ -3451,7 +3450,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "39e116df",
+   "id": "5d172bfd",
    "metadata": {
     "editable": true
    },
@@ -3468,7 +3467,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "30d2f2c3",
+   "id": "672de375",
    "metadata": {
     "editable": true
    },
@@ -3484,7 +3483,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "7e5a597b",
+   "id": "d0ed234f",
    "metadata": {
     "editable": true
    },
@@ -3496,7 +3495,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "4514d138",
+   "id": "7a2bde58",
    "metadata": {
     "editable": true
    },
@@ -3509,7 +3508,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "581de71b",
+   "id": "88344390",
    "metadata": {
     "editable": true
    },
@@ -3523,7 +3522,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "7eade20d",
+   "id": "4019cf50",
    "metadata": {
     "editable": true
    },
@@ -3533,7 +3532,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "164ca354",
+   "id": "95d6e396",
    "metadata": {
     "editable": true
    },
@@ -3546,7 +3545,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "f82f15ec",
+   "id": "d5baa57d",
    "metadata": {
     "editable": true
    },
@@ -3565,7 +3564,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "e9ecfbe0",
+   "id": "f310ebda",
    "metadata": {
     "editable": true
    },
@@ -3577,7 +3576,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "69aac1cd",
+   "id": "a8366c66",
    "metadata": {
     "editable": true
    },
@@ -3589,7 +3588,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "2055a5b8",
+   "id": "9983b7fb",
    "metadata": {
     "editable": true
    },
@@ -3599,7 +3598,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "927516d0",
+   "id": "196e9dff",
    "metadata": {
     "editable": true
    },
@@ -3611,7 +3610,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "a58a52b4",
+   "id": "fd64e6a2",
    "metadata": {
     "editable": true
    },
@@ -3624,7 +3623,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "819a1ff5",
+   "id": "46a1e1da",
    "metadata": {
     "editable": true
    },
@@ -3643,7 +3642,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "24d43e16",
+   "id": "6ea1ae21",
    "metadata": {
     "editable": true
    },
@@ -3653,7 +3652,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "7eca0f06",
+   "id": "fccc043c",
    "metadata": {
     "editable": true
    },
@@ -3672,7 +3671,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "04bf3889",
+   "id": "5d65cf6e",
    "metadata": {
     "editable": true
    },
@@ -3690,7 +3689,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "c1f805e1",
+   "id": "f6c00582",
    "metadata": {
     "editable": true
    },
@@ -3704,7 +3703,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "c696df78",
+   "id": "fd88cfa1",
    "metadata": {
     "editable": true
    },
@@ -3719,7 +3718,7 @@
   {
    "cell_type": "code",
    "execution_count": 13,
-   "id": "5f05dc7e",
+   "id": "c6da7058",
    "metadata": {
     "collapsed": false,
     "editable": true
@@ -3740,7 +3739,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "2d99bd39",
+   "id": "bd31ca44",
    "metadata": {
     "editable": true
    },
@@ -3757,7 +3756,7 @@
   {
    "cell_type": "code",
    "execution_count": 14,
-   "id": "a40890e3",
+   "id": "d809cece",
    "metadata": {
     "collapsed": false,
     "editable": true
@@ -3789,7 +3788,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "c031063b",
+   "id": "8253074c",
    "metadata": {
     "editable": true
    },
@@ -3803,7 +3802,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "f441b1cc",
+   "id": "7a74656a",
    "metadata": {
     "editable": true
    },
@@ -3816,7 +3815,7 @@
   {
    "cell_type": "code",
    "execution_count": 15,
-   "id": "e1485588",
+   "id": "804a8ee1",
    "metadata": {
     "collapsed": false,
     "editable": true
@@ -3841,7 +3840,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "e5210041",
+   "id": "3871673a",
    "metadata": {
     "editable": true
    },
@@ -3851,7 +3850,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "d97cf3fc",
+   "id": "5e75414e",
    "metadata": {
     "editable": true
    },
@@ -3862,7 +3861,7 @@
   {
    "cell_type": "code",
    "execution_count": 16,
-   "id": "19805d36",
+   "id": "b3efcdb9",
    "metadata": {
     "collapsed": false,
     "editable": true
@@ -3916,7 +3915,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "f7c176cb",
+   "id": "03523168",
    "metadata": {
     "editable": true
    },
@@ -3933,7 +3932,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "99748b66",
+   "id": "ab0fcb46",
    "metadata": {
     "editable": true
    },
@@ -3945,7 +3944,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "146a709a",
+   "id": "9e11454d",
    "metadata": {
     "editable": true
    },
@@ -3957,7 +3956,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "7bb735a1",
+   "id": "35eaf4ef",
    "metadata": {
     "editable": true
    },
@@ -3967,7 +3966,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "f28204bc",
+   "id": "a426a48c",
    "metadata": {
     "editable": true
    },
@@ -3984,7 +3983,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "d7632836",
+   "id": "f0ac2bfa",
    "metadata": {
     "editable": true
    },
@@ -3994,7 +3993,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "3b66313c",
+   "id": "c6d8c9b4",
    "metadata": {
     "editable": true
    },
@@ -4009,7 +4008,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "7970a23a",
+   "id": "a87b1d66",
    "metadata": {
     "editable": true
    },
@@ -4019,7 +4018,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "20f1bfc3",
+   "id": "cb950b85",
    "metadata": {
     "editable": true
    },
@@ -4033,7 +4032,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "5314cc68",
+   "id": "c79e96c6",
    "metadata": {
     "editable": true
    },
@@ -4045,7 +4044,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "df38126e",
+   "id": "1fceefdc",
    "metadata": {
     "editable": true
    },
@@ -4057,7 +4056,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "c0b851ee",
+   "id": "a8db1029",
    "metadata": {
     "editable": true
    },
@@ -4069,7 +4068,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "1f7f4a30",
+   "id": "03d9d476",
    "metadata": {
     "editable": true
    },
@@ -4079,7 +4078,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "096824a1",
+   "id": "9ccc8461",
    "metadata": {
     "editable": true
    },
@@ -4091,7 +4090,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "54e04c69",
+   "id": "c6183bd4",
    "metadata": {
     "editable": true
    },
@@ -4101,7 +4100,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "7e8d11af",
+   "id": "5cc84134",
    "metadata": {
     "editable": true
    },
@@ -4118,7 +4117,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "54c1dc98",
+   "id": "788d9c68",
    "metadata": {
     "editable": true
    },
@@ -4128,7 +4127,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "db51e12a",
+   "id": "bd0e5aee",
    "metadata": {
     "editable": true
    },
@@ -4140,7 +4139,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "683fad63",
+   "id": "dc52b90c",
    "metadata": {
     "editable": true
    },
@@ -4150,7 +4149,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "1b731b71",
+   "id": "b593db6e",
    "metadata": {
     "editable": true
    },
@@ -4162,7 +4161,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "2e3d21a5",
+   "id": "8dcd6cc8",
    "metadata": {
     "editable": true
    },
@@ -4176,7 +4175,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "5c62c361",
+   "id": "29312718",
    "metadata": {
     "editable": true
    },
@@ -4188,7 +4187,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "64b8d908",
+   "id": "9818fcb1",
    "metadata": {
     "editable": true
    },
@@ -4210,7 +4209,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "fd0e56a2",
+   "id": "56fbca85",
    "metadata": {
     "editable": true
    },
@@ -4222,7 +4221,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "1b0aa1e6",
+   "id": "d9d152fe",
    "metadata": {
     "editable": true
    },
@@ -4237,7 +4236,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "6924b66b",
+   "id": "b82dd67d",
    "metadata": {
     "editable": true
    },
@@ -4249,7 +4248,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "9996cc20",
+   "id": "7dab117e",
    "metadata": {
     "editable": true
    },
@@ -4261,7 +4260,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "586c291e",
+   "id": "3649c008",
    "metadata": {
     "editable": true
    },
@@ -4271,7 +4270,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "0b9dd539",
+   "id": "665e42ab",
    "metadata": {
     "editable": true
    },
@@ -4283,7 +4282,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "a8794b80",
+   "id": "254d4fd4",
    "metadata": {
     "editable": true
    },
@@ -4293,7 +4292,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "045c8882",
+   "id": "4b84bb18",
    "metadata": {
     "editable": true
    },
@@ -4305,7 +4304,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "e28bdf2f",
+   "id": "d84538dd",
    "metadata": {
     "editable": true
    },
@@ -4315,7 +4314,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "f3c04273",
+   "id": "f9a8f033",
    "metadata": {
     "editable": true
    },
@@ -4327,7 +4326,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "9a0b68ea",
+   "id": "0c900c57",
    "metadata": {
     "editable": true
    },
@@ -4344,7 +4343,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "5eed2425",
+   "id": "ef8aafcf",
    "metadata": {
     "editable": true
    },
@@ -4357,19 +4356,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "9dab0c45",
+   "id": "efc29850",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "{\\displaystyle \\min_{\\boldsymbol{\\beta}\\in {\\mathbb{R}}^{p}}}\\frac{1}{n}\\left\\{\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)\\right\\}.\n",
+    "{\\displaystyle \\min_{\\boldsymbol{\\theta}\\in {\\mathbb{R}}^{p}}}\\frac{1}{n}\\left\\{\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\right)\\right\\}.\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "5644b71a",
+   "id": "586af0fa",
    "metadata": {
     "editable": true
    },
@@ -4379,20 +4378,20 @@
   },
   {
    "cell_type": "markdown",
-   "id": "e4639fb2",
+   "id": "116d6e8b",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "{\\displaystyle \\min_{\\boldsymbol{\\beta}\\in\n",
-    "{\\mathbb{R}}^{p}}}\\frac{1}{n}\\sum_{i=0}^{n-1}\\left(y_i-\\tilde{y}_i\\right)^2=\\frac{1}{n}\\vert\\vert \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\vert\\vert_2^2,\n",
+    "{\\displaystyle \\min_{\\boldsymbol{\\theta}\\in\n",
+    "{\\mathbb{R}}^{p}}}\\frac{1}{n}\\sum_{i=0}^{n-1}\\left(y_i-\\tilde{y}_i\\right)^2=\\frac{1}{n}\\vert\\vert \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\vert\\vert_2^2,\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "5fee1177",
+   "id": "90567753",
    "metadata": {
     "editable": true
    },
@@ -4402,7 +4401,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "d536a108",
+   "id": "aa3053f1",
    "metadata": {
     "editable": true
    },
@@ -4414,57 +4413,57 @@
   },
   {
    "cell_type": "markdown",
-   "id": "33e4ced5",
+   "id": "b2382074",
    "metadata": {
     "editable": true
    },
    "source": [
     "By minimizing the above equation with respect to the parameters\n",
-    "$\\boldsymbol{\\beta}$ we could then obtain an analytical expression for the\n",
-    "parameters $\\boldsymbol{\\beta}$.  We can add a regularization parameter $\\lambda$ by\n",
+    "$\\boldsymbol{\\theta}$ we could then obtain an analytical expression for the\n",
+    "parameters $\\boldsymbol{\\theta}$.  We can add a regularization parameter $\\lambda$ by\n",
     "defining a new cost function to be optimized, that is"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "88d83772",
+   "id": "716a3c2a",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "{\\displaystyle \\min_{\\boldsymbol{\\beta}\\in\n",
-    "{\\mathbb{R}}^{p}}}\\frac{1}{n}\\vert\\vert \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\vert\\vert_2^2+\\lambda\\vert\\vert \\boldsymbol{\\beta}\\vert\\vert_2^2\n",
+    "{\\displaystyle \\min_{\\boldsymbol{\\theta}\\in\n",
+    "{\\mathbb{R}}^{p}}}\\frac{1}{n}\\vert\\vert \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\vert\\vert_2^2+\\lambda\\vert\\vert \\boldsymbol{\\theta}\\vert\\vert_2^2\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "ec444bdc",
+   "id": "dfd4ebf1",
    "metadata": {
     "editable": true
    },
    "source": [
     "which leads to the Ridge regression minimization problem where we\n",
-    "require that $\\vert\\vert \\boldsymbol{\\beta}\\vert\\vert_2^2\\le t$, where $t$ is\n",
+    "require that $\\vert\\vert \\boldsymbol{\\theta}\\vert\\vert_2^2\\le t$, where $t$ is\n",
     "a finite number larger than zero. By defining"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "968537e5",
+   "id": "d782ab23",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "C(\\boldsymbol{X},\\boldsymbol{\\beta})=\\frac{1}{n}\\vert\\vert \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\vert\\vert_2^2+\\lambda\\vert\\vert \\boldsymbol{\\beta}\\vert\\vert_1,\n",
+    "C(\\boldsymbol{X},\\boldsymbol{\\theta})=\\frac{1}{n}\\vert\\vert \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\vert\\vert_2^2+\\lambda\\vert\\vert \\boldsymbol{\\theta}\\vert\\vert_1,\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "fd16ef5d",
+   "id": "f16d590d",
    "metadata": {
     "editable": true
    },
@@ -4474,20 +4473,20 @@
   },
   {
    "cell_type": "markdown",
-   "id": "ee9bafbd",
+   "id": "a373fb74",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "{\\displaystyle \\min_{\\boldsymbol{\\beta}\\in\n",
-    "{\\mathbb{R}}^{p}}}\\frac{1}{n}\\vert\\vert \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\vert\\vert_2^2+\\lambda\\vert\\vert \\boldsymbol{\\beta}\\vert\\vert_1\n",
+    "{\\displaystyle \\min_{\\boldsymbol{\\theta}\\in\n",
+    "{\\mathbb{R}}^{p}}}\\frac{1}{n}\\vert\\vert \\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\vert\\vert_2^2+\\lambda\\vert\\vert \\boldsymbol{\\theta}\\vert\\vert_1\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "cf86de9f",
+   "id": "c4c2a6db",
    "metadata": {
     "editable": true
    },
@@ -4499,7 +4498,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "d65614ad",
+   "id": "f14edde4",
    "metadata": {
     "editable": true
    },
@@ -4511,7 +4510,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "ca3686ce",
+   "id": "899bc3ae",
    "metadata": {
     "editable": true
    },
@@ -4523,25 +4522,25 @@
   },
   {
    "cell_type": "markdown",
-   "id": "aeec861a",
+   "id": "0968804d",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "C(\\boldsymbol{X},\\boldsymbol{\\beta})=\\left\\{(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta})^T(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta})\\right\\}+\\lambda\\boldsymbol{\\beta}^T\\boldsymbol{\\beta},\n",
+    "C(\\boldsymbol{X},\\boldsymbol{\\theta})=\\left\\{(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta})^T(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta})\\right\\}+\\lambda\\boldsymbol{\\theta}^T\\boldsymbol{\\theta},\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "c0cbdf16",
+   "id": "87643992",
    "metadata": {
     "editable": true
    },
    "source": [
     "and \n",
-    "taking the derivatives with respect to $\\boldsymbol{\\beta}$ we obtain then\n",
+    "taking the derivatives with respect to $\\boldsymbol{\\theta}$ we obtain then\n",
     "a slightly modified matrix inversion problem which for finite values\n",
     "of $\\lambda$ does not suffer from singularity problems. We obtain\n",
     "the optimal parameters"
@@ -4549,19 +4548,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "f3ce9172",
+   "id": "900e9e55",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\hat{\\boldsymbol{\\beta}}_{\\mathrm{Ridge}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}+\\lambda\\boldsymbol{I}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y},\n",
+    "\\hat{\\boldsymbol{\\theta}}_{\\mathrm{Ridge}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}+\\lambda\\boldsymbol{I}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y},\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "63fb91f4",
+   "id": "6d3a5c46",
    "metadata": {
     "editable": true
    },
@@ -4571,43 +4570,43 @@
   },
   {
    "cell_type": "markdown",
-   "id": "e2e09d2c",
+   "id": "db3c68ed",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\sum_{i=0}^{p-1} \\beta_i^2 \\leq t,\n",
+    "\\sum_{i=0}^{p-1} \\theta_i^2 \\leq t,\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "51621114",
+   "id": "ab6d0410",
    "metadata": {
     "editable": true
    },
    "source": [
     "with $t$ a finite positive number. \n",
     "\n",
-    "If we keep the $1/n$ factor, the equation for the optimal $\\beta$ changes to"
+    "If we keep the $1/n$ factor, the equation for the optimal $\\theta$ changes to"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "78f80a63",
+   "id": "ae454f20",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\hat{\\boldsymbol{\\beta}}_{\\mathrm{Ridge}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}+n\\lambda\\boldsymbol{I}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y}.\n",
+    "\\hat{\\boldsymbol{\\theta}}_{\\mathrm{Ridge}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}+n\\lambda\\boldsymbol{I}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y}.\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "ac75ee51",
+   "id": "2a508398",
    "metadata": {
     "editable": true
    },
@@ -4619,19 +4618,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "474894b7",
+   "id": "6df6cf96",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\hat{\\boldsymbol{\\beta}}_{\\mathrm{OLS}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y},\n",
+    "\\hat{\\boldsymbol{\\theta}}_{\\mathrm{OLS}} = \\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y},\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "254701bf",
+   "id": "4ab38d70",
    "metadata": {
     "editable": true
    },
@@ -4642,7 +4641,7 @@
     "modified diagonal term added to $\\boldsymbol{X}^T\\boldsymbol{X}$. The consequences, in\n",
     "particular for our discussion of the bias-variance tradeoff are rather\n",
     "interesting. We will see that for specific values of $\\lambda$, we may\n",
-    "even reduce the variance of the optimal parameters $\\boldsymbol{\\beta}$. These topics and other related ones, will be discussed after the more linear algebra oriented analysis here.\n",
+    "even reduce the variance of the optimal parameters $\\boldsymbol{\\theta}$. These topics and other related ones, will be discussed after the more linear algebra oriented analysis here.\n",
     "\n",
     "Using our insights about the SVD of the design matrix $\\boldsymbol{X}$ \n",
     "We have already analyzed the OLS solutions in terms of the eigenvectors (the columns) of the right singular value matrix $\\boldsymbol{U}$ as"
@@ -4650,19 +4649,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "31fe4fa7",
+   "id": "cfa1a762",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\tilde{\\boldsymbol{y}}_{\\mathrm{OLS}}=\\boldsymbol{X}\\boldsymbol{\\beta}  =\\boldsymbol{U}\\boldsymbol{U}^T\\boldsymbol{y}.\n",
+    "\\tilde{\\boldsymbol{y}}_{\\mathrm{OLS}}=\\boldsymbol{X}\\boldsymbol{\\theta}  =\\boldsymbol{U}\\boldsymbol{U}^T\\boldsymbol{y}.\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "3a9c5112",
+   "id": "7394c1c2",
    "metadata": {
     "editable": true
    },
@@ -4672,19 +4671,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "cd0353f6",
+   "id": "c67e8228",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\tilde{\\boldsymbol{y}}_{\\mathrm{Ridge}}=\\boldsymbol{X}\\boldsymbol{\\beta}_{\\mathrm{Ridge}} = \\boldsymbol{U\\Sigma V^T}\\left(\\boldsymbol{V}\\boldsymbol{\\Sigma}^2\\boldsymbol{V}^T+\\lambda\\boldsymbol{I} \\right)^{-1}(\\boldsymbol{U\\Sigma V^T})^T\\boldsymbol{y}=\\sum_{j=0}^{p-1}\\boldsymbol{u}_j\\boldsymbol{u}_j^T\\frac{\\sigma_j^2}{\\sigma_j^2+\\lambda}\\boldsymbol{y},\n",
+    "\\tilde{\\boldsymbol{y}}_{\\mathrm{Ridge}}=\\boldsymbol{X}\\boldsymbol{\\theta}_{\\mathrm{Ridge}} = \\boldsymbol{U\\Sigma V^T}\\left(\\boldsymbol{V}\\boldsymbol{\\Sigma}^2\\boldsymbol{V}^T+\\lambda\\boldsymbol{I} \\right)^{-1}(\\boldsymbol{U\\Sigma V^T})^T\\boldsymbol{y}=\\sum_{j=0}^{p-1}\\boldsymbol{u}_j\\boldsymbol{u}_j^T\\frac{\\sigma_j^2}{\\sigma_j^2+\\lambda}\\boldsymbol{y},\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "dc9874e9",
+   "id": "95c698ae",
    "metadata": {
     "editable": true
    },
@@ -4694,7 +4693,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "11386feb",
+   "id": "372c4fae",
    "metadata": {
     "editable": true
    },
@@ -4706,7 +4705,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "affaaaf6",
+   "id": "826e010b",
    "metadata": {
     "editable": true
    },
@@ -4718,7 +4717,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "1a888182",
+   "id": "b3c2a178",
    "metadata": {
     "editable": true
    },
@@ -4734,7 +4733,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "86273ea0",
+   "id": "2ec82017",
    "metadata": {
     "editable": true
    },
@@ -4746,7 +4745,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "2d4aaac8",
+   "id": "dacb08f8",
    "metadata": {
     "editable": true
    },
@@ -4758,7 +4757,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "2bfc281a",
+   "id": "af53eaa0",
    "metadata": {
     "editable": true
    },
@@ -4768,19 +4767,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "2040777d",
+   "id": "128f95fc",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\boldsymbol{\\beta}^{\\mathrm{OLS}} = \\boldsymbol{X}^T\\boldsymbol{y}=\\sum_{i=0}^{n-1}\\boldsymbol{u}_i\\boldsymbol{u}_i^T\\boldsymbol{y},\n",
+    "\\boldsymbol{\\theta}^{\\mathrm{OLS}} = \\boldsymbol{X}^T\\boldsymbol{y}=\\sum_{i=0}^{n-1}\\boldsymbol{u}_i\\boldsymbol{u}_i^T\\boldsymbol{y},\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "a63d3173",
+   "id": "bd5b4b5d",
    "metadata": {
     "editable": true
    },
@@ -4790,19 +4789,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "271e2323",
+   "id": "d316a1ae",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\boldsymbol{\\beta}^{\\mathrm{Ridge}} = \\left(\\boldsymbol{I}+\\lambda\\boldsymbol{I}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y}=\\left(1+\\lambda\\right)^{-1}\\boldsymbol{\\beta}^{\\mathrm{OLS}},\n",
+    "\\boldsymbol{\\theta}^{\\mathrm{Ridge}} = \\left(\\boldsymbol{I}+\\lambda\\boldsymbol{I}\\right)^{-1}\\boldsymbol{X}^T\\boldsymbol{y}=\\left(1+\\lambda\\right)^{-1}\\boldsymbol{\\theta}^{\\mathrm{OLS}},\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "9e2436e2",
+   "id": "1198b684",
    "metadata": {
     "editable": true
    },
@@ -4819,7 +4818,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "17a55a55",
+   "id": "bb24826b",
    "metadata": {
     "editable": true
    },
@@ -4831,41 +4830,41 @@
   },
   {
    "cell_type": "markdown",
-   "id": "c313de2e",
+   "id": "b9e921e5",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "C(\\boldsymbol{X},\\boldsymbol{\\beta})=\\frac{1}{n}\\left\\{(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta})^T(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta})\\right\\}+\\lambda\\vert\\vert\\boldsymbol{\\beta}\\vert\\vert_1,\n",
+    "C(\\boldsymbol{X},\\boldsymbol{\\theta})=\\frac{1}{n}\\left\\{(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta})^T(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta})\\right\\}+\\lambda\\vert\\vert\\boldsymbol{\\theta}\\vert\\vert_1,\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "a8f03e7f",
+   "id": "06ad064c",
    "metadata": {
     "editable": true
    },
    "source": [
-    "Taking the derivative with respect to $\\boldsymbol{\\beta}$ and recalling that the derivative of the absolute value is (we drop the boldfaced vector symbol for simplicty)"
+    "Taking the derivative with respect to $\\boldsymbol{\\theta}$ and recalling that the derivative of the absolute value is (we drop the boldfaced vector symbol for simplicty)"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "359569a7",
+   "id": "ab63b0d3",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\frac{d \\vert \\beta\\vert}{d \\beta}=\\mathrm{sgn}(\\beta)=\\left\\{\\begin{array}{cc} 1 & \\beta > 0 \\\\-1 & \\beta < 0, \\end{array}\\right.\n",
+    "\\frac{d \\vert \\theta\\vert}{d \\theta}=\\mathrm{sgn}(\\theta)=\\left\\{\\begin{array}{cc} 1 & \\theta > 0 \\\\-1 & \\theta < 0, \\end{array}\\right.\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "76d42b3d",
+   "id": "49285a6a",
    "metadata": {
     "editable": true
    },
@@ -4875,19 +4874,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "4bb63551",
+   "id": "46be0f62",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\frac{\\partial C(\\boldsymbol{X},\\boldsymbol{\\beta})}{\\partial \\boldsymbol{\\beta}}=-\\frac{2}{n}\\boldsymbol{X}^T(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta})+\\lambda sgn(\\boldsymbol{\\beta})=0,\n",
+    "\\frac{\\partial C(\\boldsymbol{X},\\boldsymbol{\\theta})}{\\partial \\boldsymbol{\\theta}}=-\\frac{2}{n}\\boldsymbol{X}^T(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta})+\\lambda sgn(\\boldsymbol{\\theta})=0,\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "d5eb25ad",
+   "id": "1ad7dd7c",
    "metadata": {
     "editable": true
    },
@@ -4897,19 +4896,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "38afb347",
+   "id": "3d0b7ab5",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\boldsymbol{X}^T\\boldsymbol{X}\\boldsymbol{\\beta}+\\frac{n}{2}\\lambda sgn(\\boldsymbol{\\beta})=2\\boldsymbol{X}^T\\boldsymbol{y}.\n",
+    "\\boldsymbol{X}^T\\boldsymbol{X}\\boldsymbol{\\theta}+\\frac{n}{2}\\lambda sgn(\\boldsymbol{\\theta})=2\\boldsymbol{X}^T\\boldsymbol{y}.\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "687738a0",
+   "id": "b360d642",
    "metadata": {
     "editable": true
    },
@@ -4919,19 +4918,19 @@
   },
   {
    "cell_type": "markdown",
-   "id": "06c513a7",
+   "id": "8677a588",
    "metadata": {
     "editable": true
    },
    "source": [
     "$$\n",
-    "\\boldsymbol{X}^T\\boldsymbol{X}\\boldsymbol{\\beta}+\\lambda sgn(\\boldsymbol{\\beta})=2\\boldsymbol{X}^T\\boldsymbol{y}.\n",
+    "\\boldsymbol{X}^T\\boldsymbol{X}\\boldsymbol{\\theta}+\\lambda sgn(\\boldsymbol{\\theta})=2\\boldsymbol{X}^T\\boldsymbol{y}.\n",
     "$$"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "88794c16",
+   "id": "d4f0eccb",
    "metadata": {
     "editable": true
    },
diff --git a/doc/src/week35/week35.do.txt b/doc/src/week35/week35.do.txt
index e8ffa19c5..bfde18397 100644
--- a/doc/src/week35/week35.do.txt
+++ b/doc/src/week35/week35.do.txt
@@ -30,12 +30,11 @@ o For exercise 1 of week 35, the book by A. Aldo Faisal, Cheng Soon Ong, and Mar
 !split
 ===== Reminder from last week =====
 
-We need first a reminder from last week about linear regression. 
+We need first a reminder from last week about linear regression. We are going to fit a continuous function with linear parameterization in terms of the parameters  $\bm{\theta}$ and our first encounter is ordinary least squares.
 
-Fitting a continuous function with linear parameterization in terms of the parameters  $\bm{\beta}$.
-* Method of choice for fitting a continuous function!
+* It is the method of choice for fitting a continuous function
 * Gives an excellent introduction to central Machine Learning features with _understandable pedagogical_ links to other methods like _Neural Networks_, _Support Vector Machines_ etc
-* Analytical expression for the fitting parameters $\bm{\beta}$
+* Analytical expression for the fitting parameters $\bm{\theta}$
 * Analytical expressions for statistical propertiers like mean values, variances, confidence intervals and more
 * Analytical relation with probabilistic interpretations 
 * Easy to introduce basic concepts like bias-variance tradeoff, cross-validation, resampling and regularization techniques and many other ML topics
@@ -76,20 +75,20 @@ value and a variance $\sigma^2$.
 In linear regression we approximate the unknown function with another
 continuous function $\tilde{\bm{y}}(\bm{x})$ which depends linearly on
 some unknown parameters
-$\bm{\beta}^T=[\beta_0,\beta_1,\beta_2,\dots,\beta_{p-1}]$.
+$\bm{\theta}^T=[\theta_0,\theta_1,\theta_2,\dots,\theta_{p-1}]$.
 
 Last week we introduced the so-called design matrix in order to define
 the approximation $\bm{\tilde{y}}$ via the unknown quantity
-$\bm{\beta}$ as
+$\bm{\theta}$ as
 
 
 !bt
 \[
-\bm{\tilde{y}}= \bm{X}\bm{\beta},
+\bm{\tilde{y}}= \bm{X}\bm{\theta},
 \]
 !et
 
-and in order to find the optimal parameters $\beta_i$ we defined a function which
+and in order to find the optimal parameters $\theta_i$ we defined a function which
 gives a measure of the spread between the values $y_i$ (which
 represent the output values we want to reproduce) and the parametrized
 values $\tilde{y}_i$, namely the so-called cost/loss function.
@@ -101,13 +100,13 @@ values $\tilde{y}_i$, namely the so-called cost/loss function.
 We used the mean squared error to define the way we measure the quality of our model
 !bt
 \[
-C(\bm{\beta})=\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\left\{\left(\bm{y}-\bm{\tilde{y}}\right)^T\left(\bm{y}-\bm{\tilde{y}}\right)\right\},
+C(\bm{\theta})=\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\left\{\left(\bm{y}-\bm{\tilde{y}}\right)^T\left(\bm{y}-\bm{\tilde{y}}\right)\right\},
 \]
 !et
 or using the matrix $\bm{X}$ and in a more compact matrix-vector notation as
 !bt
 \[
-C(\bm{\beta})=\frac{1}{n}\left\{\left(\bm{y}-\bm{X}\bm{\beta}\right)^T\left(\bm{y}-\bm{X}\bm{\beta}\right)\right\}.
+C(\bm{\theta})=\frac{1}{n}\left\{\left(\bm{y}-\bm{X}\bm{\theta}\right)^T\left(\bm{y}-\bm{X}\bm{\theta}\right)\right\}.
 \]
 !et
 This function represents one of many possible ways to define the so-called cost function.
@@ -119,10 +118,10 @@ the function $C$ as
 
 !bt
 \[
-C(\bm{\beta})=\frac{1}{2n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2,
+C(\bm{\theta})=\frac{1}{2n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2,
 \]
 !et
-since when taking the first derivative with respect to the unknown parameters $\beta$, the factor of $2$ cancels out. 
+since when taking the first derivative with respect to the unknown parameters $\theta$, the factor of $2$ cancels out. 
 
 
 
@@ -133,14 +132,14 @@ since when taking the first derivative with respect to the unknown parameters $\
 The function 
 !bt
 \[
-C(\bm{\beta})=\frac{1}{n}\left\{\left(\bm{y}-\bm{X}\bm{\beta}\right)^T\left(\bm{y}-\bm{X}\bm{\beta}\right)\right\},
+C(\bm{\theta})=\frac{1}{n}\left\{\left(\bm{y}-\bm{X}\bm{\theta}\right)^T\left(\bm{y}-\bm{X}\bm{\theta}\right)\right\},
 \]
 !et
 can be linked to the variance of the quantity $y_i$ if we interpret the latter as the mean value. 
 When linking (see the discussions next week) with the maximum likelihood approach below, we will indeed interpret $y_i$ as a mean value
 !bt
 \[
-y_{i}=\langle y_i \rangle = \beta_0x_{i,0}+\beta_1x_{i,1}+\beta_2x_{i,2}+\dots+\beta_{n-1}x_{i,n-1}+\epsilon_i,
+y_{i}=\langle y_i \rangle = \theta_0x_{i,0}+\theta_1x_{i,1}+\theta_2x_{i,2}+\dots+\theta_{n-1}x_{i,n-1}+\epsilon_i,
 \]
 !et
 
@@ -153,29 +152,29 @@ error estimate, often limited to a statistical error estimate given by
 the standard deviation discussed earlier. In the discussion here we
 will treat $y_i$ as our exact value for the response variable.
 
-In order to find the parameters $\beta_i$ we will then minimize the spread of $C(\bm{\beta})$, that is we are going to solve the problem
+In order to find the parameters $\theta_i$ we will then minimize the spread of $C(\bm{\theta})$, that is we are going to solve the problem
 !bt
 \[
-{\displaystyle \min_{\bm{\beta}\in
-{\mathbb{R}}^{p}}}\frac{1}{n}\left\{\left(\bm{y}-\bm{X}\bm{\beta}\right)^T\left(\bm{y}-\bm{X}\bm{\beta}\right)\right\}.
+{\displaystyle \min_{\bm{\theta}\in
+{\mathbb{R}}^{p}}}\frac{1}{n}\left\{\left(\bm{y}-\bm{X}\bm{\theta}\right)^T\left(\bm{y}-\bm{X}\bm{\theta}\right)\right\}.
 \]
 !et
 In practical terms it means we will require
 !bt
 \[
-\frac{\partial C(\bm{\beta})}{\partial \beta_j} = \frac{\partial }{\partial \beta_j}\left[ \frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\beta_0x_{i,0}-\beta_1x_{i,1}-\beta_2x_{i,2}-\dots-\beta_{n-1}x_{i,n-1}\right)^2\right]=0, 
+\frac{\partial C(\bm{\theta})}{\partial \theta_j} = \frac{\partial }{\partial \theta_j}\left[ \frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\theta_0x_{i,0}-\theta_1x_{i,1}-\theta_2x_{i,2}-\dots-\theta_{n-1}x_{i,n-1}\right)^2\right]=0, 
 \]
 !et
 which results in
 !bt
 \[
-\frac{\partial C(\bm{\beta})}{\partial \beta_j} = -\frac{2}{n}\left[ \sum_{i=0}^{n-1}x_{ij}\left(y_i-\beta_0x_{i,0}-\beta_1x_{i,1}-\beta_2x_{i,2}-\dots-\beta_{n-1}x_{i,n-1}\right)\right]=0, 
+\frac{\partial C(\bm{\theta})}{\partial \theta_j} = -\frac{2}{n}\left[ \sum_{i=0}^{n-1}x_{ij}\left(y_i-\theta_0x_{i,0}-\theta_1x_{i,1}-\theta_2x_{i,2}-\dots-\theta_{n-1}x_{i,n-1}\right)\right]=0, 
 \]
 !et
 or in a matrix-vector form as (multiplying away the factor $-2/n$, see derivation below)
 !bt
 \[
-\frac{\partial C(\bm{\beta})}{\partial \bm{\beta}^T} = 0 = \bm{X}^T\left( \bm{y}-\bm{X}\bm{\beta}\right).  
+\frac{\partial C(\bm{\theta})}{\partial \bm{\theta}^T} = 0 = \bm{X}^T\left( \bm{y}-\bm{X}\bm{\theta}\right).  
 \]
 !et
 
@@ -186,19 +185,19 @@ or in a matrix-vector form as (multiplying away the factor $-2/n$, see derivatio
 We can rewrite, see the derivations below, 
 !bt
 \[
-\frac{\partial C(\bm{\beta})}{\partial \bm{\beta}^T} = 0 = \bm{X}^T\left( \bm{y}-\bm{X}\bm{\beta}\right),  
+\frac{\partial C(\bm{\theta})}{\partial \bm{\theta}^T} = 0 = \bm{X}^T\left( \bm{y}-\bm{X}\bm{\theta}\right),  
 \]
 !et
 as
 !bt
 \[
-\bm{X}^T\bm{y} = \bm{X}^T\bm{X}\bm{\beta},  
+\bm{X}^T\bm{y} = \bm{X}^T\bm{X}\bm{\theta},  
 \]
 !et
 and if the matrix $\bm{X}^T\bm{X}$ is invertible we have the solution
 !bt
 \[
-\bm{\beta} =\left(\bm{X}^T\bm{X}\right)^{-1}\bm{X}^T\bm{y}.
+\bm{\theta} =\left(\bm{X}^T\bm{X}\right)^{-1}\bm{X}^T\bm{y}.
 \]
 !et
 
@@ -420,53 +419,53 @@ and if $\bm{y}=\bm{x}$ we have
 We defined earlier a possible cost function using the mean squared error
 !bt
 \[
-C(\bm{\beta})=\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\left\{\left(\bm{y}-\bm{\tilde{y}}\right)^T\left(\bm{y}-\bm{\tilde{y}}\right)\right\},
+C(\bm{\theta})=\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\left\{\left(\bm{y}-\bm{\tilde{y}}\right)^T\left(\bm{y}-\bm{\tilde{y}}\right)\right\},
 \]
 !et
 or using the design/feature matrix $\bm{X}$ we have the more compact matrix-vector
 !bt
 \[
-C(\bm{\beta})=\frac{1}{n}\left\{\left(\bm{y}-\bm{X}\bm{\beta}\right)^T\left(\bm{y}-\bm{X}\bm{\beta}\right)\right\}.
+C(\bm{\theta})=\frac{1}{n}\left\{\left(\bm{y}-\bm{X}\bm{\theta}\right)^T\left(\bm{y}-\bm{X}\bm{\theta}\right)\right\}.
 \]
 !et
 
-We note that the design matrix $\bm{X}$ does not depend on the unknown parameters defined by the vector $\bm{\beta}$.
-We are now interested in minimizing the cost function with respect to the unknown parameters $\bm{\beta}$.
+We note that the design matrix $\bm{X}$ does not depend on the unknown parameters defined by the vector $\bm{\theta}$.
+We are now interested in minimizing the cost function with respect to the unknown parameters $\bm{\theta}$.
 
 The mean squared error is a scalar and if we use the results from example three above, we can define a new vector
 !bt
 \[
-\bm{w}=\bm{y}-\bm{X}\bm{\beta},
+\bm{w}=\bm{y}-\bm{X}\bm{\theta},
 \]
 !et
-which depends on $\bm{\beta}$. We rewrite the cost function as
+which depends on $\bm{\theta}$. We rewrite the cost function as
 !bt
 \[
-C(\bm{\beta})=\frac{1}{n}\bm{w}^T\bm{w},
+C(\bm{\theta})=\frac{1}{n}\bm{w}^T\bm{w},
 \]
 !et
 with partial derivative
 !bt
 \[
-\frac{\partial C(\bm{\beta})}{\partial \bm{\beta}}=\frac{2}{n}\bm{w}^T\frac{\partial \bm{w}}{\partial \bm{\beta}},
+\frac{\partial C(\bm{\theta})}{\partial \bm{\theta}}=\frac{2}{n}\bm{w}^T\frac{\partial \bm{w}}{\partial \bm{\theta}},
 \]
 !et
 and using that
 !bt
 \[
-\frac{\partial \bm{w}}{\partial \bm{\beta}}=-\bm{X},
+\frac{\partial \bm{w}}{\partial \bm{\theta}}=-\bm{X},
 \]
 !et
 where we used the result from example two above. Inserting the last expression we obtain
 !bt
 \[
-\frac{\partial C(\bm{\beta})}{\partial \bm{\beta}}=-\frac{2}{n}\left(\bm{y}-\bm{X}\bm{\beta}\right)^T\bm{X},
+\frac{\partial C(\bm{\theta})}{\partial \bm{\theta}}=-\frac{2}{n}\left(\bm{y}-\bm{X}\bm{\theta}\right)^T\bm{X},
 \]
 !et
 or as
 !bt
 \[
-\frac{\partial C(\bm{\beta})}{\partial \bm{\beta}^T}=-\frac{2}{n}\bm{X}^T\left(\bm{y}-\bm{X}\bm{\beta}\right).
+\frac{\partial C(\bm{\theta})}{\partial \bm{\theta}^T}=-\frac{2}{n}\bm{X}^T\left(\bm{y}-\bm{X}\bm{\theta}\right).
 \]
 !et
 
@@ -505,13 +504,13 @@ We list here some other useful relations we may encounter (recall that vectors a
 
 A very important matrix we will meet again and again in machine
 learning is the Hessian.  It is given by the second derivative of the
-cost function with respect to the parameters $\bm{\beta}$. Using the above
+cost function with respect to the parameters $\bm{\theta}$. Using the above
 expression for derivatives of vectors and matrices, we find that the
 second derivative of the mean squared error as cost function is,
 
 !bt
 \[
-\frac{\partial}{\partial \bm{\beta}}\frac{\partial C(\bm{\beta})}{\partial \bm{\beta}^T} =\frac{\partial}{\partial \bm{\beta}}\left[-\frac{2}{n}\bm{X}^T\left( \bm{y}-\bm{X}\bm{\beta}\right)\right]=\frac{2}{n}\bm{X}^T\bm{X}.  
+\frac{\partial}{\partial \bm{\theta}}\frac{\partial C(\bm{\theta})}{\partial \bm{\theta}^T} =\frac{\partial}{\partial \bm{\theta}}\left[-\frac{2}{n}\bm{X}^T\left( \bm{y}-\bm{X}\bm{\theta}\right)\right]=\frac{2}{n}\bm{X}^T\bm{X}.  
 \]
 !et
 
@@ -527,7 +526,7 @@ The Hessian matrix plays an important role and is defined here as
 
 For ordinary least squares, it is inversely proportional (derivation
 next week) with the variance of the optimal parameters
-$\hat{\bm{\beta}}$. Furthermore, we will see later this week that it is
+$\hat{\bm{\theta}}$. Furthermore, we will see later this week that it is
 (aside the factor $1/n$) equal to the covariance matrix. It plays also a very
 important role in optmization algorithms and Principal Component
 Analysis as a way to reduce the dimensionality of a machine learning/data analysis
@@ -542,22 +541,22 @@ _Linear algebra question:_  Can we use the Hessian matrix to say something about
 The residuals $\bm{\epsilon}$ are in turn given by
 !bt
 \[
-\bm{\epsilon} = \bm{y}-\bm{\tilde{y}} = \bm{y}-\bm{X}\bm{\beta},
+\bm{\epsilon} = \bm{y}-\bm{\tilde{y}} = \bm{y}-\bm{X}\bm{\theta},
 \]
 !et
 and with 
 !bt
 \[
-\bm{X}^T\left( \bm{y}-\bm{X}\bm{\beta}\right)= 0, 
+\bm{X}^T\left( \bm{y}-\bm{X}\bm{\theta}\right)= 0, 
 \]
 !et
 we have
 !bt
 \[
-\bm{X}^T\bm{\epsilon}=\bm{X}^T\left( \bm{y}-\bm{X}\bm{\beta}\right)= 0, 
+\bm{X}^T\bm{\epsilon}=\bm{X}^T\left( \bm{y}-\bm{X}\bm{\theta}\right)= 0, 
 \]
 !et
-meaning that the solution for $\bm{\beta}$ is the one which minimizes the residuals.  
+meaning that the solution for $\bm{\theta}$ is the one which minimizes the residuals.  
 
 !eblock
 
@@ -571,17 +570,17 @@ we condiser a simple polynomial fit.
 We assume our data can represented by a fourth-order polynomial. For the $i$th component we have 
 !bt
 \[
-\tilde{y}_i = \beta_0+\beta_1x_i+\beta_2x_i^2+\beta_3x_i^3+\beta_4x_i^4.
+\tilde{y}_i = \theta_0+\theta_1x_i+\theta_2x_i^2+\theta_3x_i^3+\theta_4x_i^4.
 \]
 !et
-we have five predictors/features. The first is the intercept $\beta_0$. The other terms are $\beta_i$ with $i=1,2,3,4$. Furthermore we have $n$ entries for each predictor. It means that our design matrix is an 
+we have five predictors/features. The first is the intercept $\theta_0$. The other terms are $\theta_i$ with $i=1,2,3,4$. Furthermore we have $n$ entries for each predictor. It means that our design matrix is an 
 $n\times p$ matrix $\bm{X}$.
 
 
 !split
 ===== Own code for Ordinary Least Squares =====
 
-It is rather straightforward to implement the matrix inversion and obtain the parameters $\bm{\beta}$. After having defined the matrix $\bm{X}$ and the outputs $\bm{y}$ we have 
+It is rather straightforward to implement the matrix inversion and obtain the parameters $\bm{\theta}$. After having defined the matrix $\bm{X}$ and the outputs $\bm{y}$ we have 
 !bc pycod 
 # matrix inversion to find beta
 # First we set up the data
@@ -952,21 +951,21 @@ plt.show()
 What is presented here is a mathematical analysis of various regression algorithms (ordinary least  squares, Ridge and Lasso Regression). The analysis is based on an important algorithm in linear algebra, the so-called Singular Value Decomposition (SVD). 
 
 
-We have shown that in ordinary least squares the optimal parameters $\beta$ are given by
+We have shown that in ordinary least squares the optimal parameters $\theta$ are given by
 
 !bt
 \[
-\hat{\bm{\beta}} = \left(\bm{X}^T\bm{X}\right)^{-1}\bm{X}^T\bm{y}.
+\hat{\bm{\theta}} = \left(\bm{X}^T\bm{X}\right)^{-1}\bm{X}^T\bm{y}.
 \]
 !et
 
-The _hat_ over $\bm{\beta}$ means we have the optimal parameters after minimization of the cost function.
+The _hat_ over $\bm{\theta}$ means we have the optimal parameters after minimization of the cost function.
 
 This means that our best model is defined as
 
 !bt
 \[
-\tilde{\bm{y}}=\bm{X}\hat{\bm{\beta}} = \bm{X}\left(\bm{X}^T\bm{X}\right)^{-1}\bm{X}^T\bm{y}.
+\tilde{\bm{y}}=\bm{X}\hat{\bm{\theta}} = \bm{X}\left(\bm{X}^T\bm{X}\right)^{-1}\bm{X}^T\bm{y}.
 \]
 !et
 
@@ -980,7 +979,7 @@ We now define a matrix
 We can rewrite
 !bt
 \[
-\tilde{\bm{y}}=\bm{X}\hat{\bm{\beta}} = \bm{A}\bm{y}.
+\tilde{\bm{y}}=\bm{X}\hat{\bm{\theta}} = \bm{A}\bm{y}.
 \]
 !et
 
@@ -1125,14 +1124,14 @@ This is equivalent to saying that the matrix $\bm{X}$ has at least an eigenvalue
 If our design matrix $\bm{X}$ which enters the linear regression problem
 !bt
 \begin{align}
-\bm{\beta} & =  (\bm{X}^{T} \bm{X})^{-1} \bm{X}^{T} \bm{y},
+\bm{\theta} & =  (\bm{X}^{T} \bm{X})^{-1} \bm{X}^{T} \bm{y},
 \end{align}
 !et
 has linearly dependent column vectors, we will not be able to compute the inverse
-of $\bm{X}^T\bm{X}$ and we cannot find the parameters (estimators) $\beta_i$. 
+of $\bm{X}^T\bm{X}$ and we cannot find the parameters (estimators) $\theta_i$. 
 The estimators are only well-defined if $(\bm{X}^{T}\bm{X})^{-1}$ exits. 
 This is more likely to happen when the matrix $\bm{X}$ is high-dimensional. In this case it is likely to encounter a situation where 
-the regression parameters $\beta_i$ cannot be estimated.
+the regression parameters $\theta_i$ cannot be estimated.
 
 A cheap  *ad hoc* approach is  simply to add a small diagonal component to the matrix to invert, that is we change
 !bt
@@ -1541,7 +1540,7 @@ function, that is we have
 
 !bt
 \[
-\frac{\partial^2 C(\bm{\beta})}{\partial \bm{\beta}\partial \bm{\beta}^T} =\frac{2}{n}\bm{X}^T\bm{X}.  
+\frac{\partial^2 C(\bm{\theta})}{\partial \bm{\theta}\partial \bm{\theta}^T} =\frac{2}{n}\bm{X}^T\bm{X}.  
 \]
 !et
 This quantity defines was what is called the Hessian matrix (the second derivative of a function we want to optimize).
@@ -2034,14 +2033,14 @@ Let us remind ourselves about the expression for the standard Mean Squared Error
 our optimization problem is
 !bt
 \[
-{\displaystyle \min_{\bm{\beta}\in {\mathbb{R}}^{p}}}\frac{1}{n}\left\{\left(\bm{y}-\bm{X}\bm{\beta}\right)^T\left(\bm{y}-\bm{X}\bm{\beta}\right)\right\}.
+{\displaystyle \min_{\bm{\theta}\in {\mathbb{R}}^{p}}}\frac{1}{n}\left\{\left(\bm{y}-\bm{X}\bm{\theta}\right)^T\left(\bm{y}-\bm{X}\bm{\theta}\right)\right\}.
 \]
 !et
 or we can state it as
 !bt
 \[
-{\displaystyle \min_{\bm{\beta}\in
-{\mathbb{R}}^{p}}}\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\vert\vert \bm{y}-\bm{X}\bm{\beta}\vert\vert_2^2,
+{\displaystyle \min_{\bm{\theta}\in
+{\mathbb{R}}^{p}}}\frac{1}{n}\sum_{i=0}^{n-1}\left(y_i-\tilde{y}_i\right)^2=\frac{1}{n}\vert\vert \bm{y}-\bm{X}\bm{\theta}\vert\vert_2^2,
 \]
 !et
 where we have used the definition of  a norm-2 vector, that is
@@ -2052,32 +2051,32 @@ where we have used the definition of  a norm-2 vector, that is
 !et
 
 By minimizing the above equation with respect to the parameters
-$\bm{\beta}$ we could then obtain an analytical expression for the
-parameters $\bm{\beta}$.  We can add a regularization parameter $\lambda$ by
+$\bm{\theta}$ we could then obtain an analytical expression for the
+parameters $\bm{\theta}$.  We can add a regularization parameter $\lambda$ by
 defining a new cost function to be optimized, that is
 
 !bt
 \[
-{\displaystyle \min_{\bm{\beta}\in
-{\mathbb{R}}^{p}}}\frac{1}{n}\vert\vert \bm{y}-\bm{X}\bm{\beta}\vert\vert_2^2+\lambda\vert\vert \bm{\beta}\vert\vert_2^2
+{\displaystyle \min_{\bm{\theta}\in
+{\mathbb{R}}^{p}}}\frac{1}{n}\vert\vert \bm{y}-\bm{X}\bm{\theta}\vert\vert_2^2+\lambda\vert\vert \bm{\theta}\vert\vert_2^2
 \]
 !et
 
 which leads to the Ridge regression minimization problem where we
-require that $\vert\vert \bm{\beta}\vert\vert_2^2\le t$, where $t$ is
+require that $\vert\vert \bm{\theta}\vert\vert_2^2\le t$, where $t$ is
 a finite number larger than zero. By defining
 
 !bt
 \[
-C(\bm{X},\bm{\beta})=\frac{1}{n}\vert\vert \bm{y}-\bm{X}\bm{\beta}\vert\vert_2^2+\lambda\vert\vert \bm{\beta}\vert\vert_1,
+C(\bm{X},\bm{\theta})=\frac{1}{n}\vert\vert \bm{y}-\bm{X}\bm{\theta}\vert\vert_2^2+\lambda\vert\vert \bm{\theta}\vert\vert_1,
 \]
 !et
 
 we have a new optimization equation
 !bt
 \[
-{\displaystyle \min_{\bm{\beta}\in
-{\mathbb{R}}^{p}}}\frac{1}{n}\vert\vert \bm{y}-\bm{X}\bm{\beta}\vert\vert_2^2+\lambda\vert\vert \bm{\beta}\vert\vert_1
+{\displaystyle \min_{\bm{\theta}\in
+{\mathbb{R}}^{p}}}\frac{1}{n}\vert\vert \bm{y}-\bm{X}\bm{\theta}\vert\vert_2^2+\lambda\vert\vert \bm{\theta}\vert\vert_1
 \]
 !et
 which leads to Lasso regression. Lasso stands for least absolute shrinkage and selection operator. 
@@ -2097,17 +2096,17 @@ Using the matrix-vector expression for Ridge regression and dropping the paramet
 
 !bt
 \[
-C(\bm{X},\bm{\beta})=\left\{(\bm{y}-\bm{X}\bm{\beta})^T(\bm{y}-\bm{X}\bm{\beta})\right\}+\lambda\bm{\beta}^T\bm{\beta},
+C(\bm{X},\bm{\theta})=\left\{(\bm{y}-\bm{X}\bm{\theta})^T(\bm{y}-\bm{X}\bm{\theta})\right\}+\lambda\bm{\theta}^T\bm{\theta},
 \]
 !et
 and 
-taking the derivatives with respect to $\bm{\beta}$ we obtain then
+taking the derivatives with respect to $\bm{\theta}$ we obtain then
 a slightly modified matrix inversion problem which for finite values
 of $\lambda$ does not suffer from singularity problems. We obtain
 the optimal parameters
 !bt
 \[
-\hat{\bm{\beta}}_{\mathrm{Ridge}} = \left(\bm{X}^T\bm{X}+\lambda\bm{I}\right)^{-1}\bm{X}^T\bm{y},
+\hat{\bm{\theta}}_{\mathrm{Ridge}} = \left(\bm{X}^T\bm{X}+\lambda\bm{I}\right)^{-1}\bm{X}^T\bm{y},
 \]
 !et
 
@@ -2115,16 +2114,16 @@ with $\bm{I}$ being a $p\times p$ identity matrix with the constraint that
 
 !bt
 \[
-\sum_{i=0}^{p-1} \beta_i^2 \leq t,
+\sum_{i=0}^{p-1} \theta_i^2 \leq t,
 \]
 !et
 
 with $t$ a finite positive number. 
 
-If we keep the $1/n$ factor, the equation for the optimal $\beta$ changes to
+If we keep the $1/n$ factor, the equation for the optimal $\theta$ changes to
 !bt
 \[
-\hat{\bm{\beta}}_{\mathrm{Ridge}} = \left(\bm{X}^T\bm{X}+n\lambda\bm{I}\right)^{-1}\bm{X}^T\bm{y}.
+\hat{\bm{\theta}}_{\mathrm{Ridge}} = \left(\bm{X}^T\bm{X}+n\lambda\bm{I}\right)^{-1}\bm{X}^T\bm{y}.
 \]
 !et
 
@@ -2133,7 +2132,7 @@ In many textbooks the $1/n$ term is often omitted. Note that a library like _Sci
 When we compare this with the ordinary least squares result we have
 !bt
 \[
-\hat{\bm{\beta}}_{\mathrm{OLS}} = \left(\bm{X}^T\bm{X}\right)^{-1}\bm{X}^T\bm{y},
+\hat{\bm{\theta}}_{\mathrm{OLS}} = \left(\bm{X}^T\bm{X}\right)^{-1}\bm{X}^T\bm{y},
 \]
 !et
 which can lead to singular matrices. However, with the SVD, we can always compute the inverse of the matrix $\bm{X}^T\bm{X}$.
@@ -2143,13 +2142,13 @@ We see that Ridge regression is nothing but the standard OLS with a
 modified diagonal term added to $\bm{X}^T\bm{X}$. The consequences, in
 particular for our discussion of the bias-variance tradeoff are rather
 interesting. We will see that for specific values of $\lambda$, we may
-even reduce the variance of the optimal parameters $\bm{\beta}$. These topics and other related ones, will be discussed after the more linear algebra oriented analysis here.
+even reduce the variance of the optimal parameters $\bm{\theta}$. These topics and other related ones, will be discussed after the more linear algebra oriented analysis here.
 
 Using our insights about the SVD of the design matrix $\bm{X}$ 
 We have already analyzed the OLS solutions in terms of the eigenvectors (the columns) of the right singular value matrix $\bm{U}$ as
 !bt
 \[
-\tilde{\bm{y}}_{\mathrm{OLS}}=\bm{X}\bm{\beta}  =\bm{U}\bm{U}^T\bm{y}.
+\tilde{\bm{y}}_{\mathrm{OLS}}=\bm{X}\bm{\theta}  =\bm{U}\bm{U}^T\bm{y}.
 \]
 !et
 
@@ -2158,7 +2157,7 @@ For Ridge regression this becomes
 
 !bt
 \[
-\tilde{\bm{y}}_{\mathrm{Ridge}}=\bm{X}\bm{\beta}_{\mathrm{Ridge}} = \bm{U\Sigma V^T}\left(\bm{V}\bm{\Sigma}^2\bm{V}^T+\lambda\bm{I} \right)^{-1}(\bm{U\Sigma V^T})^T\bm{y}=\sum_{j=0}^{p-1}\bm{u}_j\bm{u}_j^T\frac{\sigma_j^2}{\sigma_j^2+\lambda}\bm{y},
+\tilde{\bm{y}}_{\mathrm{Ridge}}=\bm{X}\bm{\theta}_{\mathrm{Ridge}} = \bm{U\Sigma V^T}\left(\bm{V}\bm{\Sigma}^2\bm{V}^T+\lambda\bm{I} \right)^{-1}(\bm{U\Sigma V^T})^T\bm{y}=\sum_{j=0}^{p-1}\bm{u}_j\bm{u}_j^T\frac{\sigma_j^2}{\sigma_j^2+\lambda}\bm{y},
 \]
 !et
 
@@ -2198,7 +2197,7 @@ For the sake of simplicity, let us assume that the design matrix is orthonormal,
 In this case the standard OLS results in 
 !bt
 \[
-\bm{\beta}^{\mathrm{OLS}} = \bm{X}^T\bm{y}=\sum_{i=0}^{n-1}\bm{u}_i\bm{u}_i^T\bm{y},
+\bm{\theta}^{\mathrm{OLS}} = \bm{X}^T\bm{y}=\sum_{i=0}^{n-1}\bm{u}_i\bm{u}_i^T\bm{y},
 \]
 !et
 
@@ -2206,7 +2205,7 @@ and
 
 !bt
 \[
-\bm{\beta}^{\mathrm{Ridge}} = \left(\bm{I}+\lambda\bm{I}\right)^{-1}\bm{X}^T\bm{y}=\left(1+\lambda\right)^{-1}\bm{\beta}^{\mathrm{OLS}},
+\bm{\theta}^{\mathrm{Ridge}} = \left(\bm{I}+\lambda\bm{I}\right)^{-1}\bm{X}^T\bm{y}=\left(1+\lambda\right)^{-1}\bm{\theta}^{\mathrm{OLS}},
 \]
 !et
 
@@ -2226,33 +2225,33 @@ Using the matrix-vector expression for Lasso regression, we have the following _
 
 !bt
 \[
-C(\bm{X},\bm{\beta})=\frac{1}{n}\left\{(\bm{y}-\bm{X}\bm{\beta})^T(\bm{y}-\bm{X}\bm{\beta})\right\}+\lambda\vert\vert\bm{\beta}\vert\vert_1,
+C(\bm{X},\bm{\theta})=\frac{1}{n}\left\{(\bm{y}-\bm{X}\bm{\theta})^T(\bm{y}-\bm{X}\bm{\theta})\right\}+\lambda\vert\vert\bm{\theta}\vert\vert_1,
 \]
 !et
 
-Taking the derivative with respect to $\bm{\beta}$ and recalling that the derivative of the absolute value is (we drop the boldfaced vector symbol for simplicty)
+Taking the derivative with respect to $\bm{\theta}$ and recalling that the derivative of the absolute value is (we drop the boldfaced vector symbol for simplicty)
 !bt
 \[
-\frac{d \vert \beta\vert}{d \beta}=\mathrm{sgn}(\beta)=\left\{\begin{array}{cc} 1 & \beta > 0 \\-1 & \beta < 0, \end{array}\right.
+\frac{d \vert \theta\vert}{d \theta}=\mathrm{sgn}(\theta)=\left\{\begin{array}{cc} 1 & \theta > 0 \\-1 & \theta < 0, \end{array}\right.
 \]
 !et
 we have that the derivative of the cost function is
 
 !bt
 \[
-\frac{\partial C(\bm{X},\bm{\beta})}{\partial \bm{\beta}}=-\frac{2}{n}\bm{X}^T(\bm{y}-\bm{X}\bm{\beta})+\lambda sgn(\bm{\beta})=0,
+\frac{\partial C(\bm{X},\bm{\theta})}{\partial \bm{\theta}}=-\frac{2}{n}\bm{X}^T(\bm{y}-\bm{X}\bm{\theta})+\lambda sgn(\bm{\theta})=0,
 \]
 !et
 and reordering we have
 !bt
 \[
-\bm{X}^T\bm{X}\bm{\beta}+\frac{n}{2}\lambda sgn(\bm{\beta})=2\bm{X}^T\bm{y}.
+\bm{X}^T\bm{X}\bm{\theta}+\frac{n}{2}\lambda sgn(\bm{\theta})=2\bm{X}^T\bm{y}.
 \]
 !et
 We can redefine $\lambda$ to absorb the constant $n/2$ and we rewrite the last equation as
 !bt
 \[
-\bm{X}^T\bm{X}\bm{\beta}+\lambda sgn(\bm{\beta})=2\bm{X}^T\bm{y}.
+\bm{X}^T\bm{X}\bm{\theta}+\lambda sgn(\bm{\theta})=2\bm{X}^T\bm{y}.
 \]
 !et