correcting typos

This commit is contained in:
Morten Hjorth-Jensen
2021-10-26 14:06:04 +02:00
parent bddecbcb9e
commit acf1de7ce4
10 changed files with 183 additions and 527 deletions
+1 -1
View File
@@ -331,7 +331,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Sep 28, 2021</h4></center> <!-- date -->
<center><h4>Oct 26, 2021</h4></center> <!-- date -->
<br>
<p>
+7 -29
View File
@@ -314,20 +314,6 @@ MathJax.Hub.Config({
<h2 id="further-manipulations" class="anchor">Further Manipulations </h2>
<p>
We assume
that every column of \( \boldsymbol{X} \) is centered, which we can do by subtracting the mean,
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%;"><span></span>X <span style="color: #666666">=</span> X <span style="color: #666666">-</span> np<span style="color: #666666">.</span>mean(X,axis<span style="color: #666666">=0</span>)
</pre></div>
<p>
This means that we need to rewrite \( X_{ij} \) as \( \tilde{X}_{ij}=X_{ij}-\mu_j \), where
$$
\mu_j = \frac{1}{n}\sum_{i=0}^{n-1}X_{ij}.
$$
<p>
Let us special first to the case where we have only two parameters \( \beta_0 \) and \( \beta_1 \).
Our result for \( \beta_0 \) simplifies then to
@@ -335,12 +321,12 @@ $$
n\beta_0 = \sum_{i=0}^{n-1}y_i - \sum_{i=0}^{n-1} X_{i1} \beta_1.
$$
Assuming that the matrix elements \( X_{i1} \) are centered, what we have is
We obtain then
$$
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} \left(X_{i1}-\mu_{1}\right),
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} X_{i1}.
$$
where
If we define
$$
\mu_1=\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1},
$$
@@ -352,24 +338,16 @@ $$
we have
$$
\beta_0 = \mu_y - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1}-\mu_{1}),
\beta_0 = \mu_y - \beta_1\mu_{1}.
$$
and it is easy to see that the last sum equals zero! This means that we have
In the general case, that is we have more parameters than \( \beta_0 \) and \( \beta_1 \), we have
$$
\beta_0 = \mu_y,
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \frac{1}{n}\sum_{i=0}^{n-1}\sum_{j=1}^{p-1} X_{ij}\beta_j.
$$
if the columns of the design matrix are centered. It is straight forward to generalize this results to more values of \( \beta \).
We have thus
$$
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1} y_i = \overline{\boldsymbol{y}},
$$
the average value of \( \boldsymbol{y} \).
<p>
Replacing \( y_i \) with \( y_i - \beta_0 = y_i - \overline{\boldsymbol{y}} \) and centering also our design matrix results in a cost function (in vector-matrix disguise)
Replacing \( y_i \) with \( y_i - y_i - \overline{\boldsymbol{y}} \) and centering also our design matrix results in a cost function (in vector-matrix disguise)
$$
C(\boldsymbol{\beta}) = (\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta})^T(\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta}).
$$
-2
View File
@@ -366,8 +366,6 @@ lambdas <span style="color: #666666">=</span> np<span style="color: #666666">.</
OwnRidgeBeta <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linalg<span style="color: #666666">.</span>pinv(X_train_scaled<span style="color: #666666">.</span>T <span style="color: #666666">@</span> X_train_scaled<span style="color: #666666">+</span>lmb<span style="color: #666666">*</span>I) <span style="color: #666666">@</span> X_train_scaled<span style="color: #666666">.</span>T <span style="color: #666666">@</span> (y_train_scaled)
intercept_ <span style="color: #666666">=</span> y_scaler <span style="color: #666666">-</span> X_train_mean<span style="color: #AA22FF">@OwnRidgeBeta</span> <span style="color: #408080; font-style: italic">#The intercept can be shifted so the model can predict on uncentered data</span>
<span style="color: #408080; font-style: italic">#Add intercept to prediction</span>
ypredictOwnRidge <span style="color: #666666">=</span> X_test <span style="color: #666666">@</span> OwnRidgeBeta <span style="color: #666666">+</span> intercept_
<span style="color: #408080; font-style: italic">#Add intercept to prediction</span>
ypredictOwnRidge <span style="color: #666666">=</span> X_test_scaled <span style="color: #666666">@</span> OwnRidgeBeta <span style="color: #666666">+</span> y_scaler
RegRidge <span style="color: #666666">=</span> linear_model<span style="color: #666666">.</span>Ridge(lmb)
RegRidge<span style="color: #666666">.</span>fit(X_train,y_train)
+1 -1
View File
@@ -331,7 +331,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Sep 28, 2021</h4></center> <!-- date -->
<center><h4>Oct 26, 2021</h4></center> <!-- date -->
<br>
<p>
+8 -36
View File
@@ -148,7 +148,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>&nbsp;<br>
<center><h4>Sep 28, 2021</h4></center> <!-- date -->
<center><h4>Oct 26, 2021</h4></center> <!-- date -->
<br>
<p>
@@ -556,22 +556,6 @@ $$
<section>
<h2 id="further-manipulations">Further Manipulations </h2>
<p>
We assume
that every column of \( \boldsymbol{X} \) is centered, which we can do by subtracting the mean,
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eeeedd"><pre style="font-size: 80%; line-height: 125%;"><span></span>X = X - np.mean(X,axis=<span style="color: #B452CD">0</span>)
</pre></div>
<p>
This means that we need to rewrite \( X_{ij} \) as \( \tilde{X}_{ij}=X_{ij}-\mu_j \), where
<p>&nbsp;<br>
$$
\mu_j = \frac{1}{n}\sum_{i=0}^{n-1}X_{ij}.
$$
<p>&nbsp;<br>
<p>
Let us special first to the case where we have only two parameters \( \beta_0 \) and \( \beta_1 \).
Our result for \( \beta_0 \) simplifies then to
@@ -581,14 +565,14 @@ n\beta_0 = \sum_{i=0}^{n-1}y_i - \sum_{i=0}^{n-1} X_{i1} \beta_1.
$$
<p>&nbsp;<br>
Assuming that the matrix elements \( X_{i1} \) are centered, what we have is
We obtain then
<p>&nbsp;<br>
$$
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} \left(X_{i1}-\mu_{1}\right),
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} X_{i1}.
$$
<p>&nbsp;<br>
where
If we define
<p>&nbsp;<br>
$$
\mu_1=\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1},
@@ -605,29 +589,19 @@ $$
we have
<p>&nbsp;<br>
$$
\beta_0 = \mu_y - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1}-\mu_{1}),
\beta_0 = \mu_y - \beta_1\mu_{1}.
$$
<p>&nbsp;<br>
and it is easy to see that the last sum equals zero! This means that we have
In the general case, that is we have more parameters than \( \beta_0 \) and \( \beta_1 \), we have
<p>&nbsp;<br>
$$
\beta_0 = \mu_y,
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \frac{1}{n}\sum_{i=0}^{n-1}\sum_{j=1}^{p-1} X_{ij}\beta_j.
$$
<p>&nbsp;<br>
if the columns of the design matrix are centered. It is straight forward to generalize this results to more values of \( \beta \).
We have thus
<p>&nbsp;<br>
$$
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1} y_i = \overline{\boldsymbol{y}},
$$
<p>&nbsp;<br>
the average value of \( \boldsymbol{y} \).
<p>
Replacing \( y_i \) with \( y_i - \beta_0 = y_i - \overline{\boldsymbol{y}} \) and centering also our design matrix results in a cost function (in vector-matrix disguise)
Replacing \( y_i \) with \( y_i - y_i - \overline{\boldsymbol{y}} \) and centering also our design matrix results in a cost function (in vector-matrix disguise)
<p>&nbsp;<br>
$$
C(\boldsymbol{\beta}) = (\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta})^T(\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta}).
@@ -941,8 +915,6 @@ lambdas = np.logspace(-<span style="color: #B452CD">4</span>, <span style="color
OwnRidgeBeta = np.linalg.pinv(X_train_scaled.T @ X_train_scaled+lmb*I) @ X_train_scaled.T @ (y_train_scaled)
intercept_ = y_scaler - X_train_mean<span style="color: #707a7c">@OwnRidgeBeta</span> <span style="color: #228B22">#The intercept can be shifted so the model can predict on uncentered data</span>
<span style="color: #228B22">#Add intercept to prediction</span>
ypredictOwnRidge = X_test @ OwnRidgeBeta + intercept_
<span style="color: #228B22">#Add intercept to prediction</span>
ypredictOwnRidge = X_test_scaled @ OwnRidgeBeta + y_scaler
RegRidge = linear_model.Ridge(lmb)
RegRidge.fit(X_train,y_train)
+8 -32
View File
@@ -234,7 +234,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Sep 28, 2021</h4></center> <!-- date -->
<center><h4>Oct 26, 2021</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -609,20 +609,6 @@ $$
<h2 id="further-manipulations">Further Manipulations </h2>
<p>
We assume
that every column of \( \boldsymbol{X} \) is centered, which we can do by subtracting the mean,
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eeeedd"><pre style="line-height: 125%;"><span></span>X = X - np.mean(X,axis=<span style="color: #B452CD">0</span>)
</pre></div>
<p>
This means that we need to rewrite \( X_{ij} \) as \( \tilde{X}_{ij}=X_{ij}-\mu_j \), where
$$
\mu_j = \frac{1}{n}\sum_{i=0}^{n-1}X_{ij}.
$$
<p>
Let us special first to the case where we have only two parameters \( \beta_0 \) and \( \beta_1 \).
Our result for \( \beta_0 \) simplifies then to
@@ -630,12 +616,12 @@ $$
n\beta_0 = \sum_{i=0}^{n-1}y_i - \sum_{i=0}^{n-1} X_{i1} \beta_1.
$$
Assuming that the matrix elements \( X_{i1} \) are centered, what we have is
We obtain then
$$
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} \left(X_{i1}-\mu_{1}\right),
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} X_{i1}.
$$
where
If we define
$$
\mu_1=\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1},
$$
@@ -647,24 +633,16 @@ $$
we have
$$
\beta_0 = \mu_y - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1}-\mu_{1}),
\beta_0 = \mu_y - \beta_1\mu_{1}.
$$
and it is easy to see that the last sum equals zero! This means that we have
In the general case, that is we have more parameters than \( \beta_0 \) and \( \beta_1 \), we have
$$
\beta_0 = \mu_y,
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \frac{1}{n}\sum_{i=0}^{n-1}\sum_{j=1}^{p-1} X_{ij}\beta_j.
$$
if the columns of the design matrix are centered. It is straight forward to generalize this results to more values of \( \beta \).
We have thus
$$
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1} y_i = \overline{\boldsymbol{y}},
$$
the average value of \( \boldsymbol{y} \).
<p>
Replacing \( y_i \) with \( y_i - \beta_0 = y_i - \overline{\boldsymbol{y}} \) and centering also our design matrix results in a cost function (in vector-matrix disguise)
Replacing \( y_i \) with \( y_i - y_i - \overline{\boldsymbol{y}} \) and centering also our design matrix results in a cost function (in vector-matrix disguise)
$$
C(\boldsymbol{\beta}) = (\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta})^T(\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta}).
$$
@@ -966,8 +944,6 @@ lambdas = np.logspace(-<span style="color: #B452CD">4</span>, <span style="color
OwnRidgeBeta = np.linalg.pinv(X_train_scaled.T @ X_train_scaled+lmb*I) @ X_train_scaled.T @ (y_train_scaled)
intercept_ = y_scaler - X_train_mean<span style="color: #707a7c">@OwnRidgeBeta</span> <span style="color: #228B22">#The intercept can be shifted so the model can predict on uncentered data</span>
<span style="color: #228B22">#Add intercept to prediction</span>
ypredictOwnRidge = X_test @ OwnRidgeBeta + intercept_
<span style="color: #228B22">#Add intercept to prediction</span>
ypredictOwnRidge = X_test_scaled @ OwnRidgeBeta + y_scaler
RegRidge = linear_model.Ridge(lmb)
RegRidge.fit(X_train,y_train)
+8 -32
View File
@@ -239,7 +239,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Sep 28, 2021</h4></center> <!-- date -->
<center><h4>Oct 26, 2021</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -614,20 +614,6 @@ $$
<h2 id="further-manipulations">Further Manipulations </h2>
<p>
We assume
that every column of \( \boldsymbol{X} \) is centered, which we can do by subtracting the mean,
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%;"><span></span>X <span style="color: #666666">=</span> X <span style="color: #666666">-</span> np<span style="color: #666666">.</span>mean(X,axis<span style="color: #666666">=0</span>)
</pre></div>
<p>
This means that we need to rewrite \( X_{ij} \) as \( \tilde{X}_{ij}=X_{ij}-\mu_j \), where
$$
\mu_j = \frac{1}{n}\sum_{i=0}^{n-1}X_{ij}.
$$
<p>
Let us special first to the case where we have only two parameters \( \beta_0 \) and \( \beta_1 \).
Our result for \( \beta_0 \) simplifies then to
@@ -635,12 +621,12 @@ $$
n\beta_0 = \sum_{i=0}^{n-1}y_i - \sum_{i=0}^{n-1} X_{i1} \beta_1.
$$
Assuming that the matrix elements \( X_{i1} \) are centered, what we have is
We obtain then
$$
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} \left(X_{i1}-\mu_{1}\right),
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} X_{i1}.
$$
where
If we define
$$
\mu_1=\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1},
$$
@@ -652,24 +638,16 @@ $$
we have
$$
\beta_0 = \mu_y - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1}-\mu_{1}),
\beta_0 = \mu_y - \beta_1\mu_{1}.
$$
and it is easy to see that the last sum equals zero! This means that we have
In the general case, that is we have more parameters than \( \beta_0 \) and \( \beta_1 \), we have
$$
\beta_0 = \mu_y,
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \frac{1}{n}\sum_{i=0}^{n-1}\sum_{j=1}^{p-1} X_{ij}\beta_j.
$$
if the columns of the design matrix are centered. It is straight forward to generalize this results to more values of \( \beta \).
We have thus
$$
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1} y_i = \overline{\boldsymbol{y}},
$$
the average value of \( \boldsymbol{y} \).
<p>
Replacing \( y_i \) with \( y_i - \beta_0 = y_i - \overline{\boldsymbol{y}} \) and centering also our design matrix results in a cost function (in vector-matrix disguise)
Replacing \( y_i \) with \( y_i - y_i - \overline{\boldsymbol{y}} \) and centering also our design matrix results in a cost function (in vector-matrix disguise)
$$
C(\boldsymbol{\beta}) = (\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta})^T(\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta}).
$$
@@ -971,8 +949,6 @@ lambdas <span style="color: #666666">=</span> np<span style="color: #666666">.</
OwnRidgeBeta <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linalg<span style="color: #666666">.</span>pinv(X_train_scaled<span style="color: #666666">.</span>T <span style="color: #666666">@</span> X_train_scaled<span style="color: #666666">+</span>lmb<span style="color: #666666">*</span>I) <span style="color: #666666">@</span> X_train_scaled<span style="color: #666666">.</span>T <span style="color: #666666">@</span> (y_train_scaled)
intercept_ <span style="color: #666666">=</span> y_scaler <span style="color: #666666">-</span> X_train_mean<span style="color: #AA22FF">@OwnRidgeBeta</span> <span style="color: #408080; font-style: italic">#The intercept can be shifted so the model can predict on uncentered data</span>
<span style="color: #408080; font-style: italic">#Add intercept to prediction</span>
ypredictOwnRidge <span style="color: #666666">=</span> X_test <span style="color: #666666">@</span> OwnRidgeBeta <span style="color: #666666">+</span> intercept_
<span style="color: #408080; font-style: italic">#Add intercept to prediction</span>
ypredictOwnRidge <span style="color: #666666">=</span> X_test_scaled <span style="color: #666666">@</span> OwnRidgeBeta <span style="color: #666666">+</span> y_scaler
RegRidge <span style="color: #666666">=</span> linear_model<span style="color: #666666">.</span>Ridge(lmb)
RegRidge<span style="color: #666666">.</span>fit(X_train,y_train)
Binary file not shown.
File diff suppressed because one or more lines are too long
+9 -29
View File
@@ -358,18 +358,6 @@ Multiplying away the constant $2/n$, we obtain
!split
===== Further Manipulations =====
We assume
that every column of $\bm{X}$ is centered, which we can do by subtracting the mean,
!bc pycod
X = X - np.mean(X,axis=0)
!ec
This means that we need to rewrite $X_{ij}$ as $\tilde{X}_{ij}=X_{ij}-\mu_j$, where
!bt
\[
\mu_j = \frac{1}{n}\sum_{i=0}^{n-1}X_{ij}.
\]
!et
Let us special first to the case where we have only two parameters $\beta_0$ and $\beta_1$.
Our result for $\beta_0$ simplifies then to
@@ -378,13 +366,13 @@ Our result for $\beta_0$ simplifies then to
n\beta_0 = \sum_{i=0}^{n-1}y_i - \sum_{i=0}^{n-1} X_{i1} \beta_1.
\]
!et
Assuming that the matrix elements $X_{i1}$ are centered, what we have is
We obtain then
!bt
\[
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} \left(X_{i1}-\mu_{1}\right),
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} X_{i1}.
\]
!et
where
If we define
!bt
\[
\mu_1=\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1},
@@ -399,25 +387,19 @@ and if we define the mean value of the outputs as
we have
!bt
\[
\beta_0 = \mu_y - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1}-\mu_{1}),
\beta_0 = \mu_y - \beta_1\mu_{1}.
\]
!et
and it is easy to see that the last sum equals zero! This means that we have
In the general case, that is we have more parameters than $\beta_0$ and $\beta_1$, we have
!bt
\[
\beta_0 = \mu_y,
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \frac{1}{n}\sum_{i=0}^{n-1}\sum_{j=1}^{p-1} X_{ij}\beta_j.
\]
!et
if the columns of the design matrix are centered. It is straight forward to generalize this results to more values of $\beta$.
We have thus
!bt
\[
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1} y_i = \overline{\bm{y}},
\]
!et
the average value of $\bm{y}$.
Replacing $y_i$ with $y_i - \beta_0 = y_i - \overline{\bm{y}}$ and centering also our design matrix results in a cost function (in vector-matrix disguise)
Replacing $y_i$ with $y_i - y_i - \overline{\bm{y}}$ and centering also our design matrix results in a cost function (in vector-matrix disguise)
!bt
\[
C(\boldsymbol{\beta}) = (\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta})^T(\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta}).
@@ -710,8 +692,6 @@ for i in range(nlambdas):
OwnRidgeBeta = np.linalg.pinv(X_train_scaled.T @ X_train_scaled+lmb*I) @ X_train_scaled.T @ (y_train_scaled)
intercept_ = y_scaler - X_train_mean@OwnRidgeBeta #The intercept can be shifted so the model can predict on uncentered data
#Add intercept to prediction
ypredictOwnRidge = X_test @ OwnRidgeBeta + intercept_
#Add intercept to prediction
ypredictOwnRidge = X_test_scaled @ OwnRidgeBeta + y_scaler
RegRidge = linear_model.Ridge(lmb)
RegRidge.fit(X_train,y_train)