update on notes

doc/src/week36/week36.do.txt

@@ -205,7 +205,7 @@ If our design matrix $\bm{X}$ which enters the linear regression problem
 !et
 has linearly dependent column vectors, we will not be able to compute the inverse
 of $\bm{X}^T\bm{X}$ and we cannot find the parameters (estimators) $\theta_i$. 
-The estimators are only well-defined if $(\bm{X}^{T}\bm{X})^{-1}$ exits. 
+The estimators are only well-defined if $(\bm{X}^{T}\bm{X})^{-1}$ exists. 
 This is more likely to happen when the matrix $\bm{X}$ is high-dimensional. In this case it is likely to encounter a situation where 
 the regression parameters $\theta_i$ cannot be estimated.
 
@@ -976,13 +976,13 @@ we have that the derivative of the cost function is
 and reordering we have
 !bt
 \[
-\bm{X}^T\bm{X}\bm{\theta}+\frac{n}{2}\lambda sgn(\bm{\theta})=2\bm{X}^T\bm{y}.
+\bm{X}^T\bm{X}\bm{\theta}+\frac{n}{2}\lambda sgn(\bm{\theta})=\bm{X}^T\bm{y}.
 \]
 !et
 We can redefine $\lambda$ to absorb the constant $n/2$ and we rewrite the last equation as
 !bt
 \[
-\bm{X}^T\bm{X}\bm{\theta}+\lambda sgn(\bm{\theta})=2\bm{X}^T\bm{y}.
+\bm{X}^T\bm{X}\bm{\theta}+\lambda sgn(\bm{\theta})=\bm{X}^T\bm{y}.
 \]
 !et
 
@@ -1454,30 +1454,6 @@ plt.show()
 
 !ec
 
-!split
-===== And a corresponding example using _scikit-learn_ =====
-
-!bc pycod
-# Importing various packages
-from random import random, seed
-import numpy as np
-import matplotlib.pyplot as plt
-from sklearn.linear_model import SGDRegressor
-
-n = 100
-x = 2*np.random.rand(n,1)
-y = 4+3*x+np.random.randn(n,1)
-
-X = np.c_[np.ones((n,1)), x]
-theta_linreg = np.linalg.inv(X.T @ X) @ (X.T @ y)
-print(theta_linreg)
-sgdreg = SGDRegressor(max_iter = 50, penalty=None, eta0=0.1)
-sgdreg.fit(x,y.ravel())
-print(sgdreg.intercept_, sgdreg.coef_)
-
-!ec
-
-
 
 !split 
 ===== Gradient descent and Ridge =====