correcting typos

This commit is contained in:
Morten Hjorth-Jensen
2021-10-26 14:06:04 +02:00
parent bddecbcb9e
commit acf1de7ce4
10 changed files with 183 additions and 527 deletions
+9 -29
View File
@@ -358,18 +358,6 @@ Multiplying away the constant $2/n$, we obtain
!split
===== Further Manipulations =====
We assume
that every column of $\bm{X}$ is centered, which we can do by subtracting the mean,
!bc pycod
X = X - np.mean(X,axis=0)
!ec
This means that we need to rewrite $X_{ij}$ as $\tilde{X}_{ij}=X_{ij}-\mu_j$, where
!bt
\[
\mu_j = \frac{1}{n}\sum_{i=0}^{n-1}X_{ij}.
\]
!et
Let us special first to the case where we have only two parameters $\beta_0$ and $\beta_1$.
Our result for $\beta_0$ simplifies then to
@@ -378,13 +366,13 @@ Our result for $\beta_0$ simplifies then to
n\beta_0 = \sum_{i=0}^{n-1}y_i - \sum_{i=0}^{n-1} X_{i1} \beta_1.
\]
!et
Assuming that the matrix elements $X_{i1}$ are centered, what we have is
We obtain then
!bt
\[
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} \left(X_{i1}-\mu_{1}\right),
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} X_{i1}.
\]
!et
where
If we define
!bt
\[
\mu_1=\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1},
@@ -399,25 +387,19 @@ and if we define the mean value of the outputs as
we have
!bt
\[
\beta_0 = \mu_y - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1}-\mu_{1}),
\beta_0 = \mu_y - \beta_1\mu_{1}.
\]
!et
and it is easy to see that the last sum equals zero! This means that we have
In the general case, that is we have more parameters than $\beta_0$ and $\beta_1$, we have
!bt
\[
\beta_0 = \mu_y,
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \frac{1}{n}\sum_{i=0}^{n-1}\sum_{j=1}^{p-1} X_{ij}\beta_j.
\]
!et
if the columns of the design matrix are centered. It is straight forward to generalize this results to more values of $\beta$.
We have thus
!bt
\[
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1} y_i = \overline{\bm{y}},
\]
!et
the average value of $\bm{y}$.
Replacing $y_i$ with $y_i - \beta_0 = y_i - \overline{\bm{y}}$ and centering also our design matrix results in a cost function (in vector-matrix disguise)
Replacing $y_i$ with $y_i - y_i - \overline{\bm{y}}$ and centering also our design matrix results in a cost function (in vector-matrix disguise)
!bt
\[
C(\boldsymbol{\beta}) = (\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta})^T(\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta}).
@@ -710,8 +692,6 @@ for i in range(nlambdas):
OwnRidgeBeta = np.linalg.pinv(X_train_scaled.T @ X_train_scaled+lmb*I) @ X_train_scaled.T @ (y_train_scaled)
intercept_ = y_scaler - X_train_mean@OwnRidgeBeta #The intercept can be shifted so the model can predict on uncentered data
#Add intercept to prediction
ypredictOwnRidge = X_test @ OwnRidgeBeta + intercept_
#Add intercept to prediction
ypredictOwnRidge = X_test_scaled @ OwnRidgeBeta + y_scaler
RegRidge = linear_model.Ridge(lmb)
RegRidge.fit(X_train,y_train)