-Printing the MSE, we see first that both methods give the same -MSE. However, changing the value of the intercept gives a larger or -smaller MSE, meaning that the MSE is penalized by the value of the -intercept! Setting the intercept to true in the Scikit-Learn -function or simply scaling our results, leads to a fit which is -independent of the specific value of the intercept. +Printing the MSE, we see first that both methods give the same MSE, as +they should. However, when we move to for example Ridge regression, +the way we treat the intercept may give a larger or smaller MSE, +meaning that the MSE can be penalized by the value of the +intercept. Not including the intercept in the fit, means that the +regularization term does not include \( \beta_0 \). For different values +of \( \lambda \), this may lead to differeing MSE values.
diff --git a/doc/pub/week38/html/._week38-bs015.html b/doc/pub/week38/html/._week38-bs015.html index ac68c1ea3..31ecfa924 100644 --- a/doc/pub/week38/html/._week38-bs015.html +++ b/doc/pub/week38/html/._week38-bs015.html @@ -421,7 +421,7 @@ MathJax.Hub.Config({
-Armed with this wisdom, we attempt first simply set the intercept eqault to False in our implementation of Ridge regression for yet another vanilla data set. +Armed with this wisdom, we attempt first simply set the intercept equal to False in our implementation of Ridge regression for yet another vanilla data set.
@@ -463,7 +463,6 @@ lambdas = np.for i in range(nlambdas): lmb = lambdas[i] OwnRidgeBeta = np.linalg.pinv(X_train.T @ X_train+lmb*I) @ X_train.T @ y_train - # include lasso using Scikit-Learn # Note: we include the intercept column and no scaling RegRidge = linear_model.Ridge(lmb,fit_intercept=False) RegRidge.fit(X_train,y_train) @@ -478,6 +477,11 @@ lambdas = np.print(OwnRidgeBeta) print("Beta values for Scikit-Learn Ridge implementation") print(RegRidge.coef_) + print("MSE values for own Ridge implementation") + print(MSEOwnRidgePredict[i]) + print("MSE values for Scikit-Learn Ridge implementation") + print(MSERidgePredict[i]) + # Now plot the results plt.figure() plt.plot(np.log10(lambdas), MSEOwnRidgePredict, 'r', label = 'MSE own Ridge Test') diff --git a/doc/pub/week38/html/._week38-bs016.html b/doc/pub/week38/html/._week38-bs016.html index f6be53d27..bf1accacd 100644 --- a/doc/pub/week38/html/._week38-bs016.html +++ b/doc/pub/week38/html/._week38-bs016.html @@ -478,16 +478,11 @@ lambdas = np.= y_scaler - X_train_mean@OwnRidgeBeta #The intercept can be shifted so the model can predict on uncentered data #Add intercept to prediction ypredictOwnRidge = X_test @ OwnRidgeBeta + intercept_ - #EQUIVALENT PREDICTION: #Add intercept to prediction ypredictOwnRidge = X_test_scaled @ OwnRidgeBeta + y_scaler - print("Values for own Ridge prediction") - print(ypredictOwnRidge) RegRidge = linear_model.Ridge(lmb) RegRidge.fit(X_train,y_train) ypredictRidge = RegRidge.predict(X_test) - print("Values for SL Ridge prediction") - print(ypredictRidge) MSEOwnRidgePredict[i] = MSE(y_test,ypredictOwnRidge) MSERidgePredict[i] = MSE(y_test,ypredictRidge) print("Beta values for own Ridge implementation") @@ -498,6 +493,11 @@ lambdas = np.print(intercept_) print('Intercept from Scikit-Learn Ridge implementation') print(RegRidge.intercept_) + print("MSE values for own Ridge implementation") + print(MSEOwnRidgePredict[i]) + print("MSE values for Scikit-Learn Ridge implementation") + print(MSERidgePredict[i]) + # Now plot the results plt.figure() diff --git a/doc/pub/week38/html/week38-reveal.html b/doc/pub/week38/html/week38-reveal.html index da9264ad6..604d56d16 100644 --- a/doc/pub/week38/html/week38-reveal.html +++ b/doc/pub/week38/html/week38-reveal.html @@ -771,12 +771,13 @@ The intercept is the value of our output/target variable when all our features are zero and our function crosses the \( y \)-axis (for a one-dimensional case).
-Printing the MSE, we see first that both methods give the same -MSE. However, changing the value of the intercept gives a larger or -smaller MSE, meaning that the MSE is penalized by the value of the -intercept! Setting the intercept to true in the Scikit-Learn -function or simply scaling our results, leads to a fit which is -independent of the specific value of the intercept. +Printing the MSE, we see first that both methods give the same MSE, as +they should. However, when we move to for example Ridge regression, +the way we treat the intercept may give a larger or smaller MSE, +meaning that the MSE can be penalized by the value of the +intercept. Not including the intercept in the fit, means that the +regularization term does not include \( \beta_0 \). For different values +of \( \lambda \), this may lead to differeing MSE values. @@ -784,7 +785,7 @@ independent of the specific value of the intercept.
-Armed with this wisdom, we attempt first simply set the intercept eqault to False in our implementation of Ridge regression for yet another vanilla data set. +Armed with this wisdom, we attempt first simply set the intercept equal to False in our implementation of Ridge regression for yet another vanilla data set.
@@ -826,7 +827,6 @@ lambdas = np.logspace(-4, for i in range(nlambdas): lmb = lambdas[i] OwnRidgeBeta = np.linalg.pinv(X_train.T @ X_train+lmb*I) @ X_train.T @ y_train - # include lasso using Scikit-Learn # Note: we include the intercept column and no scaling RegRidge = linear_model.Ridge(lmb,fit_intercept=False) RegRidge.fit(X_train,y_train) @@ -841,6 +841,11 @@ lambdas = np.logspace(-4, print(OwnRidgeBeta) print("Beta values for Scikit-Learn Ridge implementation") print(RegRidge.coef_) + print("MSE values for own Ridge implementation") + print(MSEOwnRidgePredict[i]) + print("MSE values for Scikit-Learn Ridge implementation") + print(MSERidgePredict[i]) + # Now plot the results plt.figure() plt.plot(np.log10(lambdas), MSEOwnRidgePredict, 'r', label = 'MSE own Ridge Test') @@ -919,16 +924,11 @@ lambdas = np.logspace(-4, @OwnRidgeBeta #The intercept can be shifted so the model can predict on uncentered data #Add intercept to prediction ypredictOwnRidge = X_test @ OwnRidgeBeta + intercept_ - #EQUIVALENT PREDICTION: #Add intercept to prediction ypredictOwnRidge = X_test_scaled @ OwnRidgeBeta + y_scaler - print("Values for own Ridge prediction") - print(ypredictOwnRidge) RegRidge = linear_model.Ridge(lmb) RegRidge.fit(X_train,y_train) ypredictRidge = RegRidge.predict(X_test) - print("Values for SL Ridge prediction") - print(ypredictRidge) MSEOwnRidgePredict[i] = MSE(y_test,ypredictOwnRidge) MSERidgePredict[i] = MSE(y_test,ypredictRidge) print("Beta values for own Ridge implementation") @@ -939,6 +939,11 @@ lambdas = np.logspace(-4, print(intercept_) print('Intercept from Scikit-Learn Ridge implementation') print(RegRidge.intercept_) + print("MSE values for own Ridge implementation") + print(MSEOwnRidgePredict[i]) + print("MSE values for Scikit-Learn Ridge implementation") + print(MSERidgePredict[i]) + # Now plot the results plt.figure() diff --git a/doc/pub/week38/html/week38-solarized.html b/doc/pub/week38/html/week38-solarized.html index c0fabbfa9..375cbd323 100644 --- a/doc/pub/week38/html/week38-solarized.html +++ b/doc/pub/week38/html/week38-solarized.html @@ -905,12 +905,13 @@ The intercept is the value of our output/target variable when all our features are zero and our function crosses the \( y \)-axis (for a one-dimensional case).
-Printing the MSE, we see first that both methods give the same -MSE. However, changing the value of the intercept gives a larger or -smaller MSE, meaning that the MSE is penalized by the value of the -intercept! Setting the intercept to true in the Scikit-Learn -function or simply scaling our results, leads to a fit which is -independent of the specific value of the intercept. +Printing the MSE, we see first that both methods give the same MSE, as +they should. However, when we move to for example Ridge regression, +the way we treat the intercept may give a larger or smaller MSE, +meaning that the MSE can be penalized by the value of the +intercept. Not including the intercept in the fit, means that the +regularization term does not include \( \beta_0 \). For different values +of \( \lambda \), this may lead to differeing MSE values.
@@ -918,7 +919,7 @@ independent of the specific value of the intercept.
-Armed with this wisdom, we attempt first simply set the intercept eqault to False in our implementation of Ridge regression for yet another vanilla data set. +Armed with this wisdom, we attempt first simply set the intercept equal to False in our implementation of Ridge regression for yet another vanilla data set.
@@ -960,7 +961,6 @@ lambdas = np.logspace(-4, for i in range(nlambdas): lmb = lambdas[i] OwnRidgeBeta = np.linalg.pinv(X_train.T @ X_train+lmb*I) @ X_train.T @ y_train - # include lasso using Scikit-Learn # Note: we include the intercept column and no scaling RegRidge = linear_model.Ridge(lmb,fit_intercept=False) RegRidge.fit(X_train,y_train) @@ -975,6 +975,11 @@ lambdas = np.logspace(-4, print(OwnRidgeBeta) print("Beta values for Scikit-Learn Ridge implementation") print(RegRidge.coef_) + print("MSE values for own Ridge implementation") + print(MSEOwnRidgePredict[i]) + print("MSE values for Scikit-Learn Ridge implementation") + print(MSERidgePredict[i]) + # Now plot the results plt.figure() plt.plot(np.log10(lambdas), MSEOwnRidgePredict, 'r', label = 'MSE own Ridge Test') @@ -1053,16 +1058,11 @@ lambdas = np.logspace(-4, @OwnRidgeBeta #The intercept can be shifted so the model can predict on uncentered data #Add intercept to prediction ypredictOwnRidge = X_test @ OwnRidgeBeta + intercept_ - #EQUIVALENT PREDICTION: #Add intercept to prediction ypredictOwnRidge = X_test_scaled @ OwnRidgeBeta + y_scaler - print("Values for own Ridge prediction") - print(ypredictOwnRidge) RegRidge = linear_model.Ridge(lmb) RegRidge.fit(X_train,y_train) ypredictRidge = RegRidge.predict(X_test) - print("Values for SL Ridge prediction") - print(ypredictRidge) MSEOwnRidgePredict[i] = MSE(y_test,ypredictOwnRidge) MSERidgePredict[i] = MSE(y_test,ypredictRidge) print("Beta values for own Ridge implementation") @@ -1073,6 +1073,11 @@ lambdas = np.logspace(-4, print(intercept_) print('Intercept from Scikit-Learn Ridge implementation') print(RegRidge.intercept_) + print("MSE values for own Ridge implementation") + print(MSEOwnRidgePredict[i]) + print("MSE values for Scikit-Learn Ridge implementation") + print(MSERidgePredict[i]) + # Now plot the results plt.figure() diff --git a/doc/pub/week38/html/week38.html b/doc/pub/week38/html/week38.html index 15a917992..14e83eaf7 100644 --- a/doc/pub/week38/html/week38.html +++ b/doc/pub/week38/html/week38.html @@ -910,12 +910,13 @@ The intercept is the value of our output/target variable when all our features are zero and our function crosses the \( y \)-axis (for a one-dimensional case).
-Printing the MSE, we see first that both methods give the same -MSE. However, changing the value of the intercept gives a larger or -smaller MSE, meaning that the MSE is penalized by the value of the -intercept! Setting the intercept to true in the Scikit-Learn -function or simply scaling our results, leads to a fit which is -independent of the specific value of the intercept. +Printing the MSE, we see first that both methods give the same MSE, as +they should. However, when we move to for example Ridge regression, +the way we treat the intercept may give a larger or smaller MSE, +meaning that the MSE can be penalized by the value of the +intercept. Not including the intercept in the fit, means that the +regularization term does not include \( \beta_0 \). For different values +of \( \lambda \), this may lead to differeing MSE values.
@@ -923,7 +924,7 @@ independent of the specific value of the intercept.
-Armed with this wisdom, we attempt first simply set the intercept eqault to False in our implementation of Ridge regression for yet another vanilla data set. +Armed with this wisdom, we attempt first simply set the intercept equal to False in our implementation of Ridge regression for yet another vanilla data set.
@@ -965,7 +966,6 @@ lambdas = np.for i in range(nlambdas): lmb = lambdas[i] OwnRidgeBeta = np.linalg.pinv(X_train.T @ X_train+lmb*I) @ X_train.T @ y_train - # include lasso using Scikit-Learn # Note: we include the intercept column and no scaling RegRidge = linear_model.Ridge(lmb,fit_intercept=False) RegRidge.fit(X_train,y_train) @@ -980,6 +980,11 @@ lambdas = np.print(OwnRidgeBeta) print("Beta values for Scikit-Learn Ridge implementation") print(RegRidge.coef_) + print("MSE values for own Ridge implementation") + print(MSEOwnRidgePredict[i]) + print("MSE values for Scikit-Learn Ridge implementation") + print(MSERidgePredict[i]) + # Now plot the results plt.figure() plt.plot(np.log10(lambdas), MSEOwnRidgePredict, 'r', label = 'MSE own Ridge Test') @@ -1058,16 +1063,11 @@ lambdas = np.= y_scaler - X_train_mean@OwnRidgeBeta #The intercept can be shifted so the model can predict on uncentered data #Add intercept to prediction ypredictOwnRidge = X_test @ OwnRidgeBeta + intercept_ - #EQUIVALENT PREDICTION: #Add intercept to prediction ypredictOwnRidge = X_test_scaled @ OwnRidgeBeta + y_scaler - print("Values for own Ridge prediction") - print(ypredictOwnRidge) RegRidge = linear_model.Ridge(lmb) RegRidge.fit(X_train,y_train) ypredictRidge = RegRidge.predict(X_test) - print("Values for SL Ridge prediction") - print(ypredictRidge) MSEOwnRidgePredict[i] = MSE(y_test,ypredictOwnRidge) MSERidgePredict[i] = MSE(y_test,ypredictRidge) print("Beta values for own Ridge implementation") @@ -1078,6 +1078,11 @@ lambdas = np.print(intercept_) print('Intercept from Scikit-Learn Ridge implementation') print(RegRidge.intercept_) + print("MSE values for own Ridge implementation") + print(MSEOwnRidgePredict[i]) + print("MSE values for Scikit-Learn Ridge implementation") + print(MSERidgePredict[i]) + # Now plot the results plt.figure() diff --git a/doc/pub/week38/ipynb/ipynb-week38-src.tar.gz b/doc/pub/week38/ipynb/ipynb-week38-src.tar.gz index 822c4fc4f..f5c873c35 100644 Binary files a/doc/pub/week38/ipynb/ipynb-week38-src.tar.gz and b/doc/pub/week38/ipynb/ipynb-week38-src.tar.gz differ diff --git a/doc/pub/week38/ipynb/week38.ipynb b/doc/pub/week38/ipynb/week38.ipynb index d6ebe80a8..bd24bc15f 100644 --- a/doc/pub/week38/ipynb/week38.ipynb +++ b/doc/pub/week38/ipynb/week38.ipynb @@ -837,17 +837,19 @@ "The intercept is the value of our output/target variable\n", "when all our features are zero and our function crosses the $y$-axis (for a one-dimensional case). \n", "\n", - "Printing the MSE, we see first that both methods give the same\n", - "MSE. However, changing the value of the intercept gives a larger or\n", - "smaller MSE, meaning that the MSE is penalized by the value of the\n", - "intercept! Setting the intercept to true in the **Scikit-Learn**\n", - "function or simply scaling our results, leads to a fit which is\n", - "independent of the specific value of the intercept.\n", + "Printing the MSE, we see first that both methods give the same MSE, as\n", + "they should. However, when we move to for example Ridge regression,\n", + "the way we treat the intercept may give a larger or smaller MSE,\n", + "meaning that the MSE can be penalized by the value of the\n", + "intercept. Not including the intercept in the fit, means that the\n", + "regularization term does not include $\\beta_0$. For different values\n", + "of $\\lambda$, this may lead to differeing MSE values.\n", + "\n", "\n", "\n", "## Code Examples\n", "\n", - "Armed with this wisdom, we attempt first simply set the intercept eqault to **False** in our implementation of Ridge regression for yet another vanilla data set." + "Armed with this wisdom, we attempt first simply set the intercept equal to **False** in our implementation of Ridge regression for yet another vanilla data set." ] }, { @@ -896,7 +898,6 @@ "for i in range(nlambdas):\n", " lmb = lambdas[i]\n", " OwnRidgeBeta = np.linalg.pinv(X_train.T @ X_train+lmb*I) @ X_train.T @ y_train\n", - " # include lasso using Scikit-Learn\n", " # Note: we include the intercept column and no scaling\n", " RegRidge = linear_model.Ridge(lmb,fit_intercept=False)\n", " RegRidge.fit(X_train,y_train)\n", @@ -911,6 +912,11 @@ " print(OwnRidgeBeta)\n", " print(\"Beta values for Scikit-Learn Ridge implementation\")\n", " print(RegRidge.coef_)\n", + " print(\"MSE values for own Ridge implementation\")\n", + " print(MSEOwnRidgePredict[i])\n", + " print(\"MSE values for Scikit-Learn Ridge implementation\")\n", + " print(MSERidgePredict[i])\n", + "\n", "# Now plot the results\n", "plt.figure()\n", "plt.plot(np.log10(lambdas), MSEOwnRidgePredict, 'r', label = 'MSE own Ridge Test')\n", @@ -998,16 +1004,11 @@ " intercept_ = y_scaler - X_train_mean@OwnRidgeBeta #The intercept can be shifted so the model can predict on uncentered data\n", " #Add intercept to prediction\n", " ypredictOwnRidge = X_test @ OwnRidgeBeta + intercept_ \n", - " #EQUIVALENT PREDICTION:\n", " #Add intercept to prediction\n", " ypredictOwnRidge = X_test_scaled @ OwnRidgeBeta + y_scaler \n", - " print(\"Values for own Ridge prediction\")\n", - " print(ypredictOwnRidge)\n", " RegRidge = linear_model.Ridge(lmb)\n", " RegRidge.fit(X_train,y_train)\n", " ypredictRidge = RegRidge.predict(X_test)\n", - " print(\"Values for SL Ridge prediction\")\n", - " print(ypredictRidge)\n", " MSEOwnRidgePredict[i] = MSE(y_test,ypredictOwnRidge)\n", " MSERidgePredict[i] = MSE(y_test,ypredictRidge)\n", " print(\"Beta values for own Ridge implementation\")\n", @@ -1018,6 +1019,11 @@ " print(intercept_)\n", " print('Intercept from Scikit-Learn Ridge implementation')\n", " print(RegRidge.intercept_)\n", + " print(\"MSE values for own Ridge implementation\")\n", + " print(MSEOwnRidgePredict[i])\n", + " print(\"MSE values for Scikit-Learn Ridge implementation\")\n", + " print(MSERidgePredict[i])\n", + "\n", "\n", "# Now plot the results\n", "plt.figure()\n", diff --git a/doc/src/week38/week38.do.txt b/doc/src/week38/week38.do.txt index 5ef70fba8..6bd895d5d 100644 --- a/doc/src/week38/week38.do.txt +++ b/doc/src/week38/week38.do.txt @@ -550,18 +550,20 @@ plt.show() The intercept is the value of our output/target variable when all our features are zero and our function crosses the $y$-axis (for a one-dimensional case). -Printing the MSE, we see first that both methods give the same -MSE. However, changing the value of the intercept gives a larger or -smaller MSE, meaning that the MSE is penalized by the value of the -intercept! Setting the intercept to true in the _Scikit-Learn_ -function or simply scaling our results, leads to a fit which is -independent of the specific value of the intercept. +Printing the MSE, we see first that both methods give the same MSE, as +they should. However, when we move to for example Ridge regression, +the way we treat the intercept may give a larger or smaller MSE, +meaning that the MSE can be penalized by the value of the +intercept. Not including the intercept in the fit, means that the +regularization term does not include $\beta_0$. For different values +of $\lambda$, this may lead to differeing MSE values. + !split ===== Code Examples ===== -Armed with this wisdom, we attempt first simply set the intercept eqault to _False_ in our implementation of Ridge regression for yet another vanilla data set. +Armed with this wisdom, we attempt first simply set the intercept equal to _False_ in our implementation of Ridge regression for yet another vanilla data set. !bc pycod import numpy as np @@ -601,7 +603,6 @@ lambdas = np.logspace(-4, 4, nlambdas) for i in range(nlambdas): lmb = lambdas[i] OwnRidgeBeta = np.linalg.pinv(X_train.T @ X_train+lmb*I) @ X_train.T @ y_train - # include lasso using Scikit-Learn # Note: we include the intercept column and no scaling RegRidge = linear_model.Ridge(lmb,fit_intercept=False) RegRidge.fit(X_train,y_train) @@ -616,6 +617,11 @@ for i in range(nlambdas): print(OwnRidgeBeta) print("Beta values for Scikit-Learn Ridge implementation") print(RegRidge.coef_) + print("MSE values for own Ridge implementation") + print(MSEOwnRidgePredict[i]) + print("MSE values for Scikit-Learn Ridge implementation") + print(MSERidgePredict[i]) + # Now plot the results plt.figure() plt.plot(np.log10(lambdas), MSEOwnRidgePredict, 'r', label = 'MSE own Ridge Test') @@ -691,16 +697,11 @@ for i in range(nlambdas): intercept_ = y_scaler - X_train_mean@OwnRidgeBeta #The intercept can be shifted so the model can predict on uncentered data #Add intercept to prediction ypredictOwnRidge = X_test @ OwnRidgeBeta + intercept_ - #EQUIVALENT PREDICTION: #Add intercept to prediction ypredictOwnRidge = X_test_scaled @ OwnRidgeBeta + y_scaler - print("Values for own Ridge prediction") - print(ypredictOwnRidge) RegRidge = linear_model.Ridge(lmb) RegRidge.fit(X_train,y_train) ypredictRidge = RegRidge.predict(X_test) - print("Values for SL Ridge prediction") - print(ypredictRidge) MSEOwnRidgePredict[i] = MSE(y_test,ypredictOwnRidge) MSERidgePredict[i] = MSE(y_test,ypredictRidge) print("Beta values for own Ridge implementation") @@ -711,6 +712,11 @@ for i in range(nlambdas): print(intercept_) print('Intercept from Scikit-Learn Ridge implementation') print(RegRidge.intercept_) + print("MSE values for own Ridge implementation") + print(MSEOwnRidgePredict[i]) + print("MSE values for Scikit-Learn Ridge implementation") + print(MSERidgePredict[i]) + # Now plot the results plt.figure()