correcting typos
This commit is contained in:
@@ -358,18 +358,6 @@ Multiplying away the constant $2/n$, we obtain
|
||||
!split
|
||||
===== Further Manipulations =====
|
||||
|
||||
We assume
|
||||
that every column of $\bm{X}$ is centered, which we can do by subtracting the mean,
|
||||
!bc pycod
|
||||
X = X - np.mean(X,axis=0)
|
||||
!ec
|
||||
|
||||
This means that we need to rewrite $X_{ij}$ as $\tilde{X}_{ij}=X_{ij}-\mu_j$, where
|
||||
!bt
|
||||
\[
|
||||
\mu_j = \frac{1}{n}\sum_{i=0}^{n-1}X_{ij}.
|
||||
\]
|
||||
!et
|
||||
|
||||
Let us special first to the case where we have only two parameters $\beta_0$ and $\beta_1$.
|
||||
Our result for $\beta_0$ simplifies then to
|
||||
@@ -378,13 +366,13 @@ Our result for $\beta_0$ simplifies then to
|
||||
n\beta_0 = \sum_{i=0}^{n-1}y_i - \sum_{i=0}^{n-1} X_{i1} \beta_1.
|
||||
\]
|
||||
!et
|
||||
Assuming that the matrix elements $X_{i1}$ are centered, what we have is
|
||||
We obtain then
|
||||
!bt
|
||||
\[
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} \left(X_{i1}-\mu_{1}\right),
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} X_{i1}.
|
||||
\]
|
||||
!et
|
||||
where
|
||||
If we define
|
||||
!bt
|
||||
\[
|
||||
\mu_1=\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1},
|
||||
@@ -399,25 +387,19 @@ and if we define the mean value of the outputs as
|
||||
we have
|
||||
!bt
|
||||
\[
|
||||
\beta_0 = \mu_y - \beta_1\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1}-\mu_{1}),
|
||||
\beta_0 = \mu_y - \beta_1\mu_{1}.
|
||||
\]
|
||||
!et
|
||||
and it is easy to see that the last sum equals zero! This means that we have
|
||||
In the general case, that is we have more parameters than $\beta_0$ and $\beta_1$, we have
|
||||
!bt
|
||||
\[
|
||||
\beta_0 = \mu_y,
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \frac{1}{n}\sum_{i=0}^{n-1}\sum_{j=1}^{p-1} X_{ij}\beta_j.
|
||||
\]
|
||||
!et
|
||||
if the columns of the design matrix are centered. It is straight forward to generalize this results to more values of $\beta$.
|
||||
We have thus
|
||||
!bt
|
||||
\[
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1} y_i = \overline{\bm{y}},
|
||||
\]
|
||||
!et
|
||||
the average value of $\bm{y}$.
|
||||
|
||||
Replacing $y_i$ with $y_i - \beta_0 = y_i - \overline{\bm{y}}$ and centering also our design matrix results in a cost function (in vector-matrix disguise)
|
||||
|
||||
|
||||
Replacing $y_i$ with $y_i - y_i - \overline{\bm{y}}$ and centering also our design matrix results in a cost function (in vector-matrix disguise)
|
||||
!bt
|
||||
\[
|
||||
C(\boldsymbol{\beta}) = (\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta})^T(\boldsymbol{\tilde{y}} - \tilde{X}\boldsymbol{\beta}).
|
||||
@@ -710,8 +692,6 @@ for i in range(nlambdas):
|
||||
OwnRidgeBeta = np.linalg.pinv(X_train_scaled.T @ X_train_scaled+lmb*I) @ X_train_scaled.T @ (y_train_scaled)
|
||||
intercept_ = y_scaler - X_train_mean@OwnRidgeBeta #The intercept can be shifted so the model can predict on uncentered data
|
||||
#Add intercept to prediction
|
||||
ypredictOwnRidge = X_test @ OwnRidgeBeta + intercept_
|
||||
#Add intercept to prediction
|
||||
ypredictOwnRidge = X_test_scaled @ OwnRidgeBeta + y_scaler
|
||||
RegRidge = linear_model.Ridge(lmb)
|
||||
RegRidge.fit(X_train,y_train)
|
||||
|
||||
Reference in New Issue
Block a user