update

doc/src/week37/exercisesweek37.do.txt

@@ -52,31 +52,33 @@ y = X.dot @ theta_true + noise
 
 This code produces a dataset where only features 0, 1, and 6
 significantly influence $\bm{y}$. The rest of the features have zero true
-coefficient, so they only contribute noise. For example, feature 0 has
+coefficient. For example, feature 0 has
 a true weight of 5.0, feature 1 has -3.0, and feature 6 has 2.0, so
 the expected relationship is:
 !bt
 \[
-y \approx 5 \times X_0 \;-\; 3 \times X_1 \;+\; 2 \times X_6 \;+\; \text{noise}.
+y \approx 5 \times x_0 \;-\; 3 \times x_1 \;+\; 2 \times x_6 \;+\; \text{noise}.
 \]
 !et
 
-
+You can remove the noise if you wish to. 
 ===== Exercise 1, scale your data =====
 
 Before fitting a regression model, it is good practice to normalize or
 standardize the features. This ensures all features are on a
 comparable scale, which is especially important when using
 regularization. Here we will perform standardization, scaling each
-feature to have mean 0 and standard deviation 1:
+feature to have mean 0 and standard deviation 1.
 
-Compute the mean and standard deviation of each column (feature) in $bm{X}$.
+=== 1a) ===
+
+Compute the mean and standard deviation of each column (feature) in $\bm{X}$.
 Subtract the mean and divide by the standard deviation for each feature.
 
 
 We will also center the target $\bm{y}$ to mean $0$. Centering $\bm{y}$
-(and each feature) means the model won’t require a separate intercept
-term – the data is shifted such that the intercept is effectively 0
+(and each feature) means the model does not require a separate intercept
+term, the data is shifted such that the intercept is effectively 0
 . (In practice, one could include an intercept in the model and not
 penalize it, but here we simplify by centering.)
 
@@ -92,17 +94,16 @@ y_mean = ?
 y_centered = ?
 !ec
 
-=== 1a) ===
 Fill in the necessary details.
 
-After this preprocessing, each column of $\bm{X}_norm$ has mean zero and standard deviation $1$
-and $\bm{y}_centered$ has mean 0. This makes the optimization landscape
+After this preprocessing, each column of $\bm{X}_{\mathrm{norm}}$ has mean zero and standard deviation $1$
+and $\bm{y}_{\mathrm{centered}}$ has mean 0. This makes the optimization landscape
 nicer and ensures the regularization penalty $\lambda \sum_j
-\beta_j^2$ treats each coefficient fairly (since features are on the
+\theta_j^2$ in Ridge regression treats each coefficient fairly (since features are on the
 same scale).
 
 
-===== Exercise 2, use the analytical formulae for OLS and Ridge regression to find the optimal paramters $\bm{theta}$ =====
+===== Exercise 2, use the analytical formulae for OLS and Ridge regression to find the optimal paramters $\bm{\theta}$ =====
 
 !bc pycod
 # Set regularization parameter, either a single value or a vector of values
@@ -117,20 +118,19 @@ print("Closed-form Ridge coefficients:", theta_closed_form)
 print("Closed-form OLS coefficients:", theta_closed_form)
 !ec
 
-This computes the ridge and OLS regression coefficients directly. The identity
-matrix $I$ has the same size as $X^T X$ (which is n_features x
-n_features), and lam * I adds $\lambda$ to the diagonal of $X^T X. We
-then invert this matrix and multiply by $X^T y. The result
-for $\bm{\theta}$  is a NumPy array of shape (n_features,) containing the
-fitted weights.
+This computes the Ridge and OLS regression coefficients directly. The identity
+matrix $I$ has the same size as $X^T X$. It adds $\lambda$ to the diagonal of $X^T X for Ridge regression. We
+then invert this matrix and multiply by $X^T y$. The result
+for $\bm{\theta}$  is a NumPy array of shape (n$\_$features,) containing the
+fitted parameters $\bm{\theta}$..
 
 === 2a) ===
-Finalize the OLS and Ridge regression determination of the optimal parameters $bm{\theta}$. 
+Finalize, in the above code, the OLS and Ridge regression determination of the optimal parameters $\bm{\theta}$. 
 
 === 2b) ===
 Explore the results as function of different values of the hyperparameter $\lambda$. See for example exercise 4 from week 36.
 
-===== Implementing the simplest form for gradient descent =====
+===== Exercise 3, Implementing the simplest form for gradient descent =====
 
 Alternatively, we can fit the ridge regression model using gradient
 descent. This is useful to visualize the iterative convergence and is
@@ -178,10 +178,10 @@ print("Gradient Descent Ridge coefficients:", theta_gdRidge)
 !ec
 
 === 3a) ===
-Discuss the results as function of the learning rate paramaters and the number of iterations.
+Discuss the results as function of the learning rate parameters and the number of iterations.
 
 === 3b) ===
-Add a stopping parameter as function of the number iterations. 
+Try to add a stopping parameter as function of the number iterations. How would you define a stopping criterion? 
 
 
 
@@ -189,5 +189,5 @@ If everything worked correctly, the learned coefficients should be
 close to the true values [5.0, -3.0, 0.0, …, 2.0, …] that we used to
 generate the data. Keep in mind that due to regularization and noise,
 the learned values will not exactly equal the true ones, but they
-should be in the same ballpark.  
+should be in the same ballpark.  Which method (OLS or Ridge) gives the best results?