update

doc/src/week42/exercisesweek42.ipynb

@@ -1,73 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "be117070",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
-    "doconce format html exercisesweek42.do.txt  -->\n",
-    "<!-- dom:TITLE: Exercises week 42 -->"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "6f7356c3",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "# Exercises week 42\n",
-    "**October 9-13, 2023**\n",
-    "\n",
-    "Date: **Deadline is Sunday October 22 at midnight**\n",
-    "\n",
-    "You can hand in the exercises from week 41 and week 42 as one exercise and get a total score of two additional points."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "aa378ef2",
-   "metadata": {
-    "editable": true
-   },
-   "source": [
-    "# Overarching aims of the exercises this week\n",
-    "\n",
-    "The aim of the exercises this week is to get started with implementing\n",
-    "gradient methods of relevance for project 2. The exercise this week is a simple\n",
-    "continuation from the  previous week with the addition of automatic differentation.\n",
-    "Everything you develop here will be used in project 2. \n",
-    "\n",
-    "In order to get started, we will now replace in our standard ordinary\n",
-    "least squares (OLS) and Ridge regression codes (from project 1) the\n",
-    "matrix inversion algorithm with our own gradient descent (GD) and SGD\n",
-    "codes.  You can use the Franke function or the terrain data from\n",
-    "project 1. **However, we recommend using a simpler function like**\n",
-    "$f(x)=a_0+a_1x+a_2x^2$ or higher-order one-dimensional polynomials.\n",
-    "You can obviously test your final codes against for example the Franke\n",
-    "function. Automatic differentiation will be discussed next week.\n",
-    "\n",
-    "You should include in your analysis of the GD and SGD codes the following elements\n",
-    "1. A plain gradient descent with a fixed learning rate (you will need to tune it) using automatic differentiation. Compare this with the analytical expression of the gradients you obtained last week. Feel free to use **Autograd** as Python package or **JAX**. You can use the examples form last week.\n",
-    "\n",
-    "2. Add momentum to the plain GD code and compare convergence with a fixed learning rate (you may need to tune the learning rate). Compare this with the analytical expression of the gradients you obtained last week.\n",
-    "\n",
-    "3. Repeat these steps for stochastic gradient descent with mini batches and a given number of epochs. Use a tunable learning rate as discussed in the lectures from week 39. Discuss the results as functions of the various parameters (size of batches, number of epochs etc)\n",
-    "\n",
-    "4. Implement the Adagrad method in order to tune the learning rate. Do this with and without momentum for plain gradient descent and SGD using automatic differentiation..\n",
-    "\n",
-    "5. Add RMSprop and Adam to your library of methods for tuning the learning rate. Again using automatic differentiation.\n",
-    "\n",
-    "The lecture notes from weeks 39 and 40 contain more information and code examples. Feel free to use these examples.\n",
-    "\n",
-    "We recommend reading chapter 8 on optimization from the textbook of [Goodfellow, Bengio and Courville](https://www.deeplearningbook.org/). This chapter contains many useful insights and discussions on the optimization part of machine learning."
-   ]
-  }
- ],
- "metadata": {},
- "nbformat": 4,
- "nbformat_minor": 5
-}

doc/src/week42/week42.do.txt

@@ -1,6 +1,6 @@
 TITLE: Week 42 Constructing a Neural Network code with introduction to Tensor flow
 AUTHOR: Morten Hjorth-Jensen {copyright, 1999-present|CC BY-NC} at Department of Physics, University of Oslo & Department of Physics and Astronomy and Facility for Rare Isotope Beams, Michigan State University
-DATE: Week 42
+DATE: October 16-20, 2023
 
 
 
@@ -29,7 +29,7 @@ I also  recommend Michael Nielsen's intuitive approach to the neural networks an
 
 
 !split
-===== Lecture Thursday October 12 =====
+===== Lecture Thursday October 19 =====
 
 
 !split