update

2023-10-09 06:14:29 +02:00
parent 3c553e703e
commit 664dce7f53
3 changed files with 2246 additions and 8 deletions
@@ -5,27 +5,27 @@ DATE: Deadline is Sunday October 15 at midnight

 =====  Overarching aims of the exercises this week =====

-The aim of the exercises this week 
-
+The aim of the exercises this week is to get started with implementing
+gradient methods of relevance for project 2. This exercise will also
+be continued next week with the addition of automatic differentation.
+Everything you develop here will be used in project 2. 

 In order to get started, we will now replace in our standard ordinary
 least squares (OLS) and Ridge regression codes (from project 1) the
 matrix inversion algorithm with our own gradient descent (GD) and SGD
 codes.  You can use the Franke function or the terrain data from
-project 1. However, we recommend using a simpler function like
+project 1. _However, we recommend using a simpler function like_
 $f(x)=a_0+a_1x+a_2x^2$ or higher-order one-dimensional polynomials.
 You can obviously test your final codes against for example the Franke
 function.

-
 You should include in your analysis of the GD and SGD codes the following elements
-o A plain gradient descent with a fixed learning rate (you will need to tune it).
-o Add momentum to the plain GD code and compare convergence with a fixed learning rate (you may need to tune the learning rate).
+o A plain gradient descent with a fixed learning rate (you will need to tune it) using the analytical expression of the gradients
+o Add momentum to the plain GD code and compare convergence with a fixed learning rate (you may need to tune the learning rate), again using the analytical expression of the gradients.
 o Repeat these steps for stochastic gradient descent with mini batches and a given number of epochs. Use a tunable learning rate as discussed in the lectures from week 39. Discuss the results as functions of the various parameters (size of batches, number of epochs etc)
 o Implement the Adagrad method in order to tune the learning rate. Do this with and without momentum for plain gradient descent and SGD.
 o Add RMSprop and Adam to your library of methods for tuning the learning rate.
-The lecture notes from "weeks 39 and 40 contain more
-details":"https://compphysics.github.io/MachineLearning/doc/pub/week39/html/week39.html" and code examples. Feel free to use these examples.
+The lecture notes from "weeks 39 and 40 contain more information and code examples. Feel free to use these examples.

 In summary, you should 
 perform an analysis of the results for OLS and Ridge regression as