correcting typos
This commit is contained in:
@@ -331,7 +331,7 @@ MathJax.Hub.Config({
|
||||
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
|
||||
<br>
|
||||
<p>
|
||||
<center><h4>Sep 28, 2021</h4></center> <!-- date -->
|
||||
<center><h4>Oct 26, 2021</h4></center> <!-- date -->
|
||||
<br>
|
||||
<p>
|
||||
|
||||
|
||||
@@ -314,20 +314,6 @@ MathJax.Hub.Config({
|
||||
|
||||
<h2 id="further-manipulations" class="anchor">Further Manipulations </h2>
|
||||
|
||||
<p>
|
||||
We assume
|
||||
that every column of \( \boldsymbol{X} \) is centered, which we can do by subtracting the mean,
|
||||
<p>
|
||||
|
||||
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
|
||||
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%;"><span></span>X <span style="color: #666666">=</span> X <span style="color: #666666">-</span> np<span style="color: #666666">.</span>mean(X,axis<span style="color: #666666">=0</span>)
|
||||
</pre></div>
|
||||
<p>
|
||||
This means that we need to rewrite \( X_{ij} \) as \( \tilde{X}_{ij}=X_{ij}-\mu_j \), where
|
||||
$$
|
||||
\mu_j = \frac{1}{n}\sum_{i=0}^{n-1}X_{ij}.
|
||||
$$
|
||||
|
||||
<p>
|
||||
Let us special first to the case where we have only two parameters \( \beta_0 \) and \( \beta_1 \).
|
||||
Our result for \( \beta_0 \) simplifies then to
|
||||
@@ -335,12 +321,12 @@ $$
|
||||
n\beta_0 = \sum_{i=0}^{n-1}y_i - \sum_{i=0}^{n-1} X_{i1} \beta_1.
|
||||
$$
|
||||
|
||||
Assuming that the matrix elements \( X_{i1} \) are centered, what we have is
|
||||
We obtain then
|
||||
$$
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} \left(X_{i1}-\mu_{1}\right),
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} X_{i1}.
|
||||
$$
|
||||
|
||||
where
|
||||
If we define
|
||||
$$
|
||||
\mu_1=\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1},
|
||||
$$
|
||||
@@ -352,24 +338,16 @@ $$
|
||||
|
||||
we have
|
||||
$$
|
||||
\beta_0 = \mu_y - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1}-\mu_{1}),
|
||||
\beta_0 = \mu_y - \beta_1\mu_{1}.
|
||||
$$
|
||||
|
||||
and it is easy to see that the last sum equals zero! This means that we have
|
||||
In the general case, that is we have more parameters than \( \beta_0 \) and \( \beta_1 \), we have
|
||||
$$
|
||||
\beta_0 = \mu_y,
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \frac{1}{n}\sum_{i=0}^{n-1}\sum_{j=1}^{p-1} X_{ij}\beta_j.
|
||||
$$
|
||||
|
||||
if the columns of the design matrix are centered. It is straight forward to generalize this results to more values of \( \beta \).
|
||||
We have thus
|
||||
$$
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1} y_i = \overline{\boldsymbol{y}},
|
||||
$$
|
||||
|
||||
the average value of \( \boldsymbol{y} \).
|
||||
|
||||
<p>
|
||||
Replacing \( y_i \) with \( y_i - \beta_0 = y_i - \overline{\boldsymbol{y}} \) and centering also our design matrix results in a cost function (in vector-matrix disguise)
|
||||
Replacing \( y_i \) with \( y_i - y_i - \overline{\boldsymbol{y}} \) and centering also our design matrix results in a cost function (in vector-matrix disguise)
|
||||
$$
|
||||
C(\boldsymbol{\beta}) = (\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta})^T(\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta}).
|
||||
$$
|
||||
|
||||
@@ -366,8 +366,6 @@ lambdas <span style="color: #666666">=</span> np<span style="color: #666666">.</
|
||||
OwnRidgeBeta <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linalg<span style="color: #666666">.</span>pinv(X_train_scaled<span style="color: #666666">.</span>T <span style="color: #666666">@</span> X_train_scaled<span style="color: #666666">+</span>lmb<span style="color: #666666">*</span>I) <span style="color: #666666">@</span> X_train_scaled<span style="color: #666666">.</span>T <span style="color: #666666">@</span> (y_train_scaled)
|
||||
intercept_ <span style="color: #666666">=</span> y_scaler <span style="color: #666666">-</span> X_train_mean<span style="color: #AA22FF">@OwnRidgeBeta</span> <span style="color: #408080; font-style: italic">#The intercept can be shifted so the model can predict on uncentered data</span>
|
||||
<span style="color: #408080; font-style: italic">#Add intercept to prediction</span>
|
||||
ypredictOwnRidge <span style="color: #666666">=</span> X_test <span style="color: #666666">@</span> OwnRidgeBeta <span style="color: #666666">+</span> intercept_
|
||||
<span style="color: #408080; font-style: italic">#Add intercept to prediction</span>
|
||||
ypredictOwnRidge <span style="color: #666666">=</span> X_test_scaled <span style="color: #666666">@</span> OwnRidgeBeta <span style="color: #666666">+</span> y_scaler
|
||||
RegRidge <span style="color: #666666">=</span> linear_model<span style="color: #666666">.</span>Ridge(lmb)
|
||||
RegRidge<span style="color: #666666">.</span>fit(X_train,y_train)
|
||||
|
||||
@@ -331,7 +331,7 @@ MathJax.Hub.Config({
|
||||
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
|
||||
<br>
|
||||
<p>
|
||||
<center><h4>Sep 28, 2021</h4></center> <!-- date -->
|
||||
<center><h4>Oct 26, 2021</h4></center> <!-- date -->
|
||||
<br>
|
||||
<p>
|
||||
|
||||
|
||||
@@ -148,7 +148,7 @@ MathJax.Hub.Config({
|
||||
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
|
||||
<br>
|
||||
<p> <br>
|
||||
<center><h4>Sep 28, 2021</h4></center> <!-- date -->
|
||||
<center><h4>Oct 26, 2021</h4></center> <!-- date -->
|
||||
<br>
|
||||
<p>
|
||||
|
||||
@@ -556,22 +556,6 @@ $$
|
||||
<section>
|
||||
<h2 id="further-manipulations">Further Manipulations </h2>
|
||||
|
||||
<p>
|
||||
We assume
|
||||
that every column of \( \boldsymbol{X} \) is centered, which we can do by subtracting the mean,
|
||||
<p>
|
||||
|
||||
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
|
||||
<div class="highlight" style="background: #eeeedd"><pre style="font-size: 80%; line-height: 125%;"><span></span>X = X - np.mean(X,axis=<span style="color: #B452CD">0</span>)
|
||||
</pre></div>
|
||||
<p>
|
||||
This means that we need to rewrite \( X_{ij} \) as \( \tilde{X}_{ij}=X_{ij}-\mu_j \), where
|
||||
<p> <br>
|
||||
$$
|
||||
\mu_j = \frac{1}{n}\sum_{i=0}^{n-1}X_{ij}.
|
||||
$$
|
||||
<p> <br>
|
||||
|
||||
<p>
|
||||
Let us special first to the case where we have only two parameters \( \beta_0 \) and \( \beta_1 \).
|
||||
Our result for \( \beta_0 \) simplifies then to
|
||||
@@ -581,14 +565,14 @@ n\beta_0 = \sum_{i=0}^{n-1}y_i - \sum_{i=0}^{n-1} X_{i1} \beta_1.
|
||||
$$
|
||||
<p> <br>
|
||||
|
||||
Assuming that the matrix elements \( X_{i1} \) are centered, what we have is
|
||||
We obtain then
|
||||
<p> <br>
|
||||
$$
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} \left(X_{i1}-\mu_{1}\right),
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} X_{i1}.
|
||||
$$
|
||||
<p> <br>
|
||||
|
||||
where
|
||||
If we define
|
||||
<p> <br>
|
||||
$$
|
||||
\mu_1=\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1},
|
||||
@@ -605,29 +589,19 @@ $$
|
||||
we have
|
||||
<p> <br>
|
||||
$$
|
||||
\beta_0 = \mu_y - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1}-\mu_{1}),
|
||||
\beta_0 = \mu_y - \beta_1\mu_{1}.
|
||||
$$
|
||||
<p> <br>
|
||||
|
||||
and it is easy to see that the last sum equals zero! This means that we have
|
||||
In the general case, that is we have more parameters than \( \beta_0 \) and \( \beta_1 \), we have
|
||||
<p> <br>
|
||||
$$
|
||||
\beta_0 = \mu_y,
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \frac{1}{n}\sum_{i=0}^{n-1}\sum_{j=1}^{p-1} X_{ij}\beta_j.
|
||||
$$
|
||||
<p> <br>
|
||||
|
||||
if the columns of the design matrix are centered. It is straight forward to generalize this results to more values of \( \beta \).
|
||||
We have thus
|
||||
<p> <br>
|
||||
$$
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1} y_i = \overline{\boldsymbol{y}},
|
||||
$$
|
||||
<p> <br>
|
||||
|
||||
the average value of \( \boldsymbol{y} \).
|
||||
|
||||
<p>
|
||||
Replacing \( y_i \) with \( y_i - \beta_0 = y_i - \overline{\boldsymbol{y}} \) and centering also our design matrix results in a cost function (in vector-matrix disguise)
|
||||
Replacing \( y_i \) with \( y_i - y_i - \overline{\boldsymbol{y}} \) and centering also our design matrix results in a cost function (in vector-matrix disguise)
|
||||
<p> <br>
|
||||
$$
|
||||
C(\boldsymbol{\beta}) = (\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta})^T(\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta}).
|
||||
@@ -941,8 +915,6 @@ lambdas = np.logspace(-<span style="color: #B452CD">4</span>, <span style="color
|
||||
OwnRidgeBeta = np.linalg.pinv(X_train_scaled.T @ X_train_scaled+lmb*I) @ X_train_scaled.T @ (y_train_scaled)
|
||||
intercept_ = y_scaler - X_train_mean<span style="color: #707a7c">@OwnRidgeBeta</span> <span style="color: #228B22">#The intercept can be shifted so the model can predict on uncentered data</span>
|
||||
<span style="color: #228B22">#Add intercept to prediction</span>
|
||||
ypredictOwnRidge = X_test @ OwnRidgeBeta + intercept_
|
||||
<span style="color: #228B22">#Add intercept to prediction</span>
|
||||
ypredictOwnRidge = X_test_scaled @ OwnRidgeBeta + y_scaler
|
||||
RegRidge = linear_model.Ridge(lmb)
|
||||
RegRidge.fit(X_train,y_train)
|
||||
|
||||
@@ -234,7 +234,7 @@ MathJax.Hub.Config({
|
||||
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
|
||||
<br>
|
||||
<p>
|
||||
<center><h4>Sep 28, 2021</h4></center> <!-- date -->
|
||||
<center><h4>Oct 26, 2021</h4></center> <!-- date -->
|
||||
<br>
|
||||
<p>
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
@@ -609,20 +609,6 @@ $$
|
||||
|
||||
<h2 id="further-manipulations">Further Manipulations </h2>
|
||||
|
||||
<p>
|
||||
We assume
|
||||
that every column of \( \boldsymbol{X} \) is centered, which we can do by subtracting the mean,
|
||||
<p>
|
||||
|
||||
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
|
||||
<div class="highlight" style="background: #eeeedd"><pre style="line-height: 125%;"><span></span>X = X - np.mean(X,axis=<span style="color: #B452CD">0</span>)
|
||||
</pre></div>
|
||||
<p>
|
||||
This means that we need to rewrite \( X_{ij} \) as \( \tilde{X}_{ij}=X_{ij}-\mu_j \), where
|
||||
$$
|
||||
\mu_j = \frac{1}{n}\sum_{i=0}^{n-1}X_{ij}.
|
||||
$$
|
||||
|
||||
<p>
|
||||
Let us special first to the case where we have only two parameters \( \beta_0 \) and \( \beta_1 \).
|
||||
Our result for \( \beta_0 \) simplifies then to
|
||||
@@ -630,12 +616,12 @@ $$
|
||||
n\beta_0 = \sum_{i=0}^{n-1}y_i - \sum_{i=0}^{n-1} X_{i1} \beta_1.
|
||||
$$
|
||||
|
||||
Assuming that the matrix elements \( X_{i1} \) are centered, what we have is
|
||||
We obtain then
|
||||
$$
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} \left(X_{i1}-\mu_{1}\right),
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} X_{i1}.
|
||||
$$
|
||||
|
||||
where
|
||||
If we define
|
||||
$$
|
||||
\mu_1=\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1},
|
||||
$$
|
||||
@@ -647,24 +633,16 @@ $$
|
||||
|
||||
we have
|
||||
$$
|
||||
\beta_0 = \mu_y - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1}-\mu_{1}),
|
||||
\beta_0 = \mu_y - \beta_1\mu_{1}.
|
||||
$$
|
||||
|
||||
and it is easy to see that the last sum equals zero! This means that we have
|
||||
In the general case, that is we have more parameters than \( \beta_0 \) and \( \beta_1 \), we have
|
||||
$$
|
||||
\beta_0 = \mu_y,
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \frac{1}{n}\sum_{i=0}^{n-1}\sum_{j=1}^{p-1} X_{ij}\beta_j.
|
||||
$$
|
||||
|
||||
if the columns of the design matrix are centered. It is straight forward to generalize this results to more values of \( \beta \).
|
||||
We have thus
|
||||
$$
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1} y_i = \overline{\boldsymbol{y}},
|
||||
$$
|
||||
|
||||
the average value of \( \boldsymbol{y} \).
|
||||
|
||||
<p>
|
||||
Replacing \( y_i \) with \( y_i - \beta_0 = y_i - \overline{\boldsymbol{y}} \) and centering also our design matrix results in a cost function (in vector-matrix disguise)
|
||||
Replacing \( y_i \) with \( y_i - y_i - \overline{\boldsymbol{y}} \) and centering also our design matrix results in a cost function (in vector-matrix disguise)
|
||||
$$
|
||||
C(\boldsymbol{\beta}) = (\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta})^T(\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta}).
|
||||
$$
|
||||
@@ -966,8 +944,6 @@ lambdas = np.logspace(-<span style="color: #B452CD">4</span>, <span style="color
|
||||
OwnRidgeBeta = np.linalg.pinv(X_train_scaled.T @ X_train_scaled+lmb*I) @ X_train_scaled.T @ (y_train_scaled)
|
||||
intercept_ = y_scaler - X_train_mean<span style="color: #707a7c">@OwnRidgeBeta</span> <span style="color: #228B22">#The intercept can be shifted so the model can predict on uncentered data</span>
|
||||
<span style="color: #228B22">#Add intercept to prediction</span>
|
||||
ypredictOwnRidge = X_test @ OwnRidgeBeta + intercept_
|
||||
<span style="color: #228B22">#Add intercept to prediction</span>
|
||||
ypredictOwnRidge = X_test_scaled @ OwnRidgeBeta + y_scaler
|
||||
RegRidge = linear_model.Ridge(lmb)
|
||||
RegRidge.fit(X_train,y_train)
|
||||
|
||||
@@ -239,7 +239,7 @@ MathJax.Hub.Config({
|
||||
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
|
||||
<br>
|
||||
<p>
|
||||
<center><h4>Sep 28, 2021</h4></center> <!-- date -->
|
||||
<center><h4>Oct 26, 2021</h4></center> <!-- date -->
|
||||
<br>
|
||||
<p>
|
||||
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
||||
@@ -614,20 +614,6 @@ $$
|
||||
|
||||
<h2 id="further-manipulations">Further Manipulations </h2>
|
||||
|
||||
<p>
|
||||
We assume
|
||||
that every column of \( \boldsymbol{X} \) is centered, which we can do by subtracting the mean,
|
||||
<p>
|
||||
|
||||
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
|
||||
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%;"><span></span>X <span style="color: #666666">=</span> X <span style="color: #666666">-</span> np<span style="color: #666666">.</span>mean(X,axis<span style="color: #666666">=0</span>)
|
||||
</pre></div>
|
||||
<p>
|
||||
This means that we need to rewrite \( X_{ij} \) as \( \tilde{X}_{ij}=X_{ij}-\mu_j \), where
|
||||
$$
|
||||
\mu_j = \frac{1}{n}\sum_{i=0}^{n-1}X_{ij}.
|
||||
$$
|
||||
|
||||
<p>
|
||||
Let us special first to the case where we have only two parameters \( \beta_0 \) and \( \beta_1 \).
|
||||
Our result for \( \beta_0 \) simplifies then to
|
||||
@@ -635,12 +621,12 @@ $$
|
||||
n\beta_0 = \sum_{i=0}^{n-1}y_i - \sum_{i=0}^{n-1} X_{i1} \beta_1.
|
||||
$$
|
||||
|
||||
Assuming that the matrix elements \( X_{i1} \) are centered, what we have is
|
||||
We obtain then
|
||||
$$
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} \left(X_{i1}-\mu_{1}\right),
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} X_{i1}.
|
||||
$$
|
||||
|
||||
where
|
||||
If we define
|
||||
$$
|
||||
\mu_1=\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1},
|
||||
$$
|
||||
@@ -652,24 +638,16 @@ $$
|
||||
|
||||
we have
|
||||
$$
|
||||
\beta_0 = \mu_y - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1}-\mu_{1}),
|
||||
\beta_0 = \mu_y - \beta_1\mu_{1}.
|
||||
$$
|
||||
|
||||
and it is easy to see that the last sum equals zero! This means that we have
|
||||
In the general case, that is we have more parameters than \( \beta_0 \) and \( \beta_1 \), we have
|
||||
$$
|
||||
\beta_0 = \mu_y,
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \frac{1}{n}\sum_{i=0}^{n-1}\sum_{j=1}^{p-1} X_{ij}\beta_j.
|
||||
$$
|
||||
|
||||
if the columns of the design matrix are centered. It is straight forward to generalize this results to more values of \( \beta \).
|
||||
We have thus
|
||||
$$
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1} y_i = \overline{\boldsymbol{y}},
|
||||
$$
|
||||
|
||||
the average value of \( \boldsymbol{y} \).
|
||||
|
||||
<p>
|
||||
Replacing \( y_i \) with \( y_i - \beta_0 = y_i - \overline{\boldsymbol{y}} \) and centering also our design matrix results in a cost function (in vector-matrix disguise)
|
||||
Replacing \( y_i \) with \( y_i - y_i - \overline{\boldsymbol{y}} \) and centering also our design matrix results in a cost function (in vector-matrix disguise)
|
||||
$$
|
||||
C(\boldsymbol{\beta}) = (\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta})^T(\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta}).
|
||||
$$
|
||||
@@ -971,8 +949,6 @@ lambdas <span style="color: #666666">=</span> np<span style="color: #666666">.</
|
||||
OwnRidgeBeta <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linalg<span style="color: #666666">.</span>pinv(X_train_scaled<span style="color: #666666">.</span>T <span style="color: #666666">@</span> X_train_scaled<span style="color: #666666">+</span>lmb<span style="color: #666666">*</span>I) <span style="color: #666666">@</span> X_train_scaled<span style="color: #666666">.</span>T <span style="color: #666666">@</span> (y_train_scaled)
|
||||
intercept_ <span style="color: #666666">=</span> y_scaler <span style="color: #666666">-</span> X_train_mean<span style="color: #AA22FF">@OwnRidgeBeta</span> <span style="color: #408080; font-style: italic">#The intercept can be shifted so the model can predict on uncentered data</span>
|
||||
<span style="color: #408080; font-style: italic">#Add intercept to prediction</span>
|
||||
ypredictOwnRidge <span style="color: #666666">=</span> X_test <span style="color: #666666">@</span> OwnRidgeBeta <span style="color: #666666">+</span> intercept_
|
||||
<span style="color: #408080; font-style: italic">#Add intercept to prediction</span>
|
||||
ypredictOwnRidge <span style="color: #666666">=</span> X_test_scaled <span style="color: #666666">@</span> OwnRidgeBeta <span style="color: #666666">+</span> y_scaler
|
||||
RegRidge <span style="color: #666666">=</span> linear_model<span style="color: #666666">.</span>Ridge(lmb)
|
||||
RegRidge<span style="color: #666666">.</span>fit(X_train,y_train)
|
||||
|
||||
Binary file not shown.
+141
-365
File diff suppressed because one or more lines are too long
@@ -358,18 +358,6 @@ Multiplying away the constant $2/n$, we obtain
|
||||
!split
|
||||
===== Further Manipulations =====
|
||||
|
||||
We assume
|
||||
that every column of $\bm{X}$ is centered, which we can do by subtracting the mean,
|
||||
!bc pycod
|
||||
X = X - np.mean(X,axis=0)
|
||||
!ec
|
||||
|
||||
This means that we need to rewrite $X_{ij}$ as $\tilde{X}_{ij}=X_{ij}-\mu_j$, where
|
||||
!bt
|
||||
\[
|
||||
\mu_j = \frac{1}{n}\sum_{i=0}^{n-1}X_{ij}.
|
||||
\]
|
||||
!et
|
||||
|
||||
Let us special first to the case where we have only two parameters $\beta_0$ and $\beta_1$.
|
||||
Our result for $\beta_0$ simplifies then to
|
||||
@@ -378,13 +366,13 @@ Our result for $\beta_0$ simplifies then to
|
||||
n\beta_0 = \sum_{i=0}^{n-1}y_i - \sum_{i=0}^{n-1} X_{i1} \beta_1.
|
||||
\]
|
||||
!et
|
||||
Assuming that the matrix elements $X_{i1}$ are centered, what we have is
|
||||
We obtain then
|
||||
!bt
|
||||
\[
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} \left(X_{i1}-\mu_{1}\right),
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} X_{i1}.
|
||||
\]
|
||||
!et
|
||||
where
|
||||
If we define
|
||||
!bt
|
||||
\[
|
||||
\mu_1=\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1},
|
||||
@@ -399,25 +387,19 @@ and if we define the mean value of the outputs as
|
||||
we have
|
||||
!bt
|
||||
\[
|
||||
\beta_0 = \mu_y - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1}-\mu_{1}),
|
||||
\beta_0 = \mu_y - \beta_1\mu_{1}.
|
||||
\]
|
||||
!et
|
||||
and it is easy to see that the last sum equals zero! This means that we have
|
||||
In the general case, that is we have more parameters than $\beta_0$ and $\beta_1$, we have
|
||||
!bt
|
||||
\[
|
||||
\beta_0 = \mu_y,
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \frac{1}{n}\sum_{i=0}^{n-1}\sum_{j=1}^{p-1} X_{ij}\beta_j.
|
||||
\]
|
||||
!et
|
||||
if the columns of the design matrix are centered. It is straight forward to generalize this results to more values of $\beta$.
|
||||
We have thus
|
||||
!bt
|
||||
\[
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1} y_i = \overline{\bm{y}},
|
||||
\]
|
||||
!et
|
||||
the average value of $\bm{y}$.
|
||||
|
||||
Replacing $y_i$ with $y_i - \beta_0 = y_i - \overline{\bm{y}}$ and centering also our design matrix results in a cost function (in vector-matrix disguise)
|
||||
|
||||
|
||||
Replacing $y_i$ with $y_i - y_i - \overline{\bm{y}}$ and centering also our design matrix results in a cost function (in vector-matrix disguise)
|
||||
!bt
|
||||
\[
|
||||
C(\boldsymbol{\beta}) = (\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta})^T(\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta}).
|
||||
@@ -710,8 +692,6 @@ for i in range(nlambdas):
|
||||
OwnRidgeBeta = np.linalg.pinv(X_train_scaled.T @ X_train_scaled+lmb*I) @ X_train_scaled.T @ (y_train_scaled)
|
||||
intercept_ = y_scaler - X_train_mean@OwnRidgeBeta #The intercept can be shifted so the model can predict on uncentered data
|
||||
#Add intercept to prediction
|
||||
ypredictOwnRidge = X_test @ OwnRidgeBeta + intercept_
|
||||
#Add intercept to prediction
|
||||
ypredictOwnRidge = X_test_scaled @ OwnRidgeBeta + y_scaler
|
||||
RegRidge = linear_model.Ridge(lmb)
|
||||
RegRidge.fit(X_train,y_train)
|
||||
|
||||
Reference in New Issue
Block a user