From 6cf5b46ca0ec5d70941426110aaa47a9f6970f76 Mon Sep 17 00:00:00 2001 From: Morten Hjorth-Jensen Date: Mon, 13 Oct 2025 08:41:59 +0200 Subject: [PATCH] update project 2 --- .../2025/Project2/html/._Project2-bs000.html | 22 +- .../2025/Project2/html/Project2-bs.html | 22 +- doc/Projects/2025/Project2/html/Project2.html | 22 +- .../2025/Project2/ipynb/Project2.ipynb | 64 +- .../Project2/ipynb/ipynb-Project2-src.tar.gz | Bin 193 -> 193 bytes doc/Projects/2025/Project2/pdf/Project2.p.tex | 18 +- doc/Projects/2025/Project2/pdf/Project2.tex | 18 +- .../Projects/2025/Project2/Project2.do.txt | 11 +- doc/src/Projects/2025/Project2/Project2.ipynb | 594 ------------------ 9 files changed, 88 insertions(+), 683 deletions(-) delete mode 100644 doc/src/Projects/2025/Project2/Project2.ipynb diff --git a/doc/Projects/2025/Project2/html/._Project2-bs000.html b/doc/Projects/2025/Project2/html/._Project2-bs000.html index 0f4bcf402..eb694ab7a 100644 --- a/doc/Projects/2025/Project2/html/._Project2-bs000.html +++ b/doc/Projects/2025/Project2/html/._Project2-bs000.html @@ -243,14 +243,14 @@ feed-forward neural network (FFNN) code. The exercises from week 41 and 42 (see

The data sets that we propose here are (the default sets)

We will start with a regression problem and we will reuse our codes on gradient descent methods from project 1.

Part a): Analytical warm-up

@@ -260,10 +260,10 @@ gradients. The functions whose gradients we need are:

  1. The mean-squared error (MSE) with and without the \( L_1 \) and \( L_2 \) norms (regression problems)
    2. -
  2. The binary cross entropy (aka log loss) for classification problems with and without \( L_1 \) and \( L_2 \) norms
    3. +
  3. The binary cross entropy (aka log loss) for binary classification problems with and without \( L_1 \) and \( L_2 \) norms
  4. The multiclass cross entropy cost/loss function (aka Softmax cross entropy or just Softmax loss function)
-

Set up these three cost/loss functions and their respective derivatives and explain the various terms.

+

Set up these three cost/loss functions and their respective derivatives and explain the various terms. In this project you will however only use the MSE and the Softmax cross entropy.

We will test three activation functions for our neural network setup, these are the

    @@ -330,7 +330,7 @@ and two hidden layers using \( 50 \) and \( 100 \) hidden nodes, respectively.

    Comment your results and give a critical discussion of the results obtained with the OLS code from project 1 and your own neural network code. Make an analysis of the learning rates employed to find the -optimal MSE and \( R2 \) scores. Test both stochastic gradient descent +optimal MSE score. Test both stochastic gradient descent with RMSprop and ADAM and plain gradient descent with different learning rates.

    diff --git a/doc/Projects/2025/Project2/html/Project2-bs.html b/doc/Projects/2025/Project2/html/Project2-bs.html index 0f4bcf402..eb694ab7a 100644 --- a/doc/Projects/2025/Project2/html/Project2-bs.html +++ b/doc/Projects/2025/Project2/html/Project2-bs.html @@ -243,14 +243,14 @@ feed-forward neural network (FFNN) code. The exercises from week 41 and 42 (see

    The data sets that we propose here are (the default sets)

    We will start with a regression problem and we will reuse our codes on gradient descent methods from project 1.

    Part a): Analytical warm-up

    @@ -260,10 +260,10 @@ gradients. The functions whose gradients we need are:

    1. The mean-squared error (MSE) with and without the \( L_1 \) and \( L_2 \) norms (regression problems)
      2. -
    2. The binary cross entropy (aka log loss) for classification problems with and without \( L_1 \) and \( L_2 \) norms
      3. +
    3. The binary cross entropy (aka log loss) for binary classification problems with and without \( L_1 \) and \( L_2 \) norms
    4. The multiclass cross entropy cost/loss function (aka Softmax cross entropy or just Softmax loss function)
    -

    Set up these three cost/loss functions and their respective derivatives and explain the various terms.

    +

    Set up these three cost/loss functions and their respective derivatives and explain the various terms. In this project you will however only use the MSE and the Softmax cross entropy.

    We will test three activation functions for our neural network setup, these are the

      @@ -330,7 +330,7 @@ and two hidden layers using \( 50 \) and \( 100 \) hidden nodes, respectively.

      Comment your results and give a critical discussion of the results obtained with the OLS code from project 1 and your own neural network code. Make an analysis of the learning rates employed to find the -optimal MSE and \( R2 \) scores. Test both stochastic gradient descent +optimal MSE score. Test both stochastic gradient descent with RMSprop and ADAM and plain gradient descent with different learning rates.

      diff --git a/doc/Projects/2025/Project2/html/Project2.html b/doc/Projects/2025/Project2/html/Project2.html index dd007d442..774e6e630 100644 --- a/doc/Projects/2025/Project2/html/Project2.html +++ b/doc/Projects/2025/Project2/html/Project2.html @@ -278,14 +278,14 @@ feed-forward neural network (FFNN) code. The exercises from week 41 and 42 (see

      The data sets that we propose here are (the default sets)

        -
      • Regression (fitting a continuous function). In this part you will need to bring back your results from project 1 and compare these with what you get from your Neural Network code to be developed here. The data sets could be -
          -
        1. The simple one-dimensional function Runge function from project 1, that is \( f(x) = \frac{1}{1+25x^2} \). We recommend using a simpler function when developing your neural network code for regression problems. You should however feel free to discuss and study other functions, such as the the two-dimensional Runge function \( f(x,y)=\left[(10x - 5)^2 + (10y - 5)^2 + 1 \right]^{-1} \), or even more complicated two-dimensional functions (see the supplementary material of https://www.nature.com/articles/s41467-025-61362-4 for an extensive list of two-dimensional functions).
          2. -
        -
      • Classification. -
          -
        1. We will consider the multiclass classification problem given by the full MNIST data set. The one included in scikit-learn is reduced data. The full data set is at https://www.kaggle.com/datasets/hojjatk/mnist-dataset.
          2. -
        +
      • Regression (fitting a continuous function). In this part you will need to bring back your results from project 1 and compare these with what you get from your Neural Network code to be developed here. The data sets could be
        • +
          +
        • The simple one-dimensional function Runge function from project 1, that is \( f(x) = \frac{1}{1+25x^2} \). We recommend using a simpler function when developing your neural network code for regression problems. Feel however free to discuss and study other functions, such as the the two-dimensional Runge function \( f(x,y)=\left[(10x - 5)^2 + (10y - 5)^2 + 1 \right]^{-1} \), or even more complicated two-dimensional functions (see the supplementary material of https://www.nature.com/articles/s41467-025-61362-4 for an extensive list of two-dimensional functions).
          • +
        +
      • Classification.
        • +

      We will start with a regression problem and we will reuse our codes on gradient descent methods from project 1.

      Part a): Analytical warm-up

      @@ -295,10 +295,10 @@ gradients. The functions whose gradients we need are:

      1. The mean-squared error (MSE) with and without the \( L_1 \) and \( L_2 \) norms (regression problems)
        2. -
      2. The binary cross entropy (aka log loss) for classification problems with and without \( L_1 \) and \( L_2 \) norms
        3. +
      3. The binary cross entropy (aka log loss) for binary classification problems with and without \( L_1 \) and \( L_2 \) norms
      4. The multiclass cross entropy cost/loss function (aka Softmax cross entropy or just Softmax loss function)
      -

      Set up these three cost/loss functions and their respective derivatives and explain the various terms.

      +

      Set up these three cost/loss functions and their respective derivatives and explain the various terms. In this project you will however only use the MSE and the Softmax cross entropy.

      We will test three activation functions for our neural network setup, these are the

        @@ -365,7 +365,7 @@ and two hidden layers using \( 50 \) and \( 100 \) hidden nodes, respectively.

        Comment your results and give a critical discussion of the results obtained with the OLS code from project 1 and your own neural network code. Make an analysis of the learning rates employed to find the -optimal MSE and \( R2 \) scores. Test both stochastic gradient descent +optimal MSE score. Test both stochastic gradient descent with RMSprop and ADAM and plain gradient descent with different learning rates.

        diff --git a/doc/Projects/2025/Project2/ipynb/Project2.ipynb b/doc/Projects/2025/Project2/ipynb/Project2.ipynb index 8b59d4d83..b22fbdd58 100644 --- a/doc/Projects/2025/Project2/ipynb/Project2.ipynb +++ b/doc/Projects/2025/Project2/ipynb/Project2.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "markdown", - "id": "3fa6e27c", + "id": "061af572", "metadata": { "editable": true }, @@ -14,7 +14,7 @@ }, { "cell_type": "markdown", - "id": "a03c6301", + "id": "22578683", "metadata": { "editable": true }, @@ -27,7 +27,7 @@ }, { "cell_type": "markdown", - "id": "cb555c06", + "id": "61fb162f", "metadata": { "editable": true }, @@ -58,7 +58,7 @@ }, { "cell_type": "markdown", - "id": "a1606b1d", + "id": "104c69e1", "metadata": { "editable": true }, @@ -104,7 +104,7 @@ }, { "cell_type": "markdown", - "id": "6cb2d855", + "id": "0d2c42e3", "metadata": { "editable": true }, @@ -121,18 +121,18 @@ "\n", "* Regression (fitting a continuous function). In this part you will need to bring back your results from project 1 and compare these with what you get from your Neural Network code to be developed here. The data sets could be\n", "\n", - "a. The simple one-dimensional function Runge function from project 1, that is $f(x) = \\frac{1}{1+25x^2}$. We recommend using a simpler function when developing your neural network code for regression problems. You should however feel free to discuss and study other functions, such as the the two-dimensional Runge function $f(x,y)=\\left[(10x - 5)^2 + (10y - 5)^2 + 1 \\right]^{-1}$, or even more complicated two-dimensional functions (see the supplementary material of for an extensive list of two-dimensional functions). \n", + " * The simple one-dimensional function Runge function from project 1, that is $f(x) = \\frac{1}{1+25x^2}$. We recommend using a simpler function when developing your neural network code for regression problems. Feel however free to discuss and study other functions, such as the the two-dimensional Runge function $f(x,y)=\\left[(10x - 5)^2 + (10y - 5)^2 + 1 \\right]^{-1}$, or even more complicated two-dimensional functions (see the supplementary material of for an extensive list of two-dimensional functions). \n", "\n", "* Classification.\n", "\n", - "a. We will consider the multiclass classification problem given by the full MNIST data set. The one included in **scikit-learn** is reduced data. The full data set is at . \n", + " * We will consider a multiclass classification problem given by the full MNIST data set. The full data set is at .\n", "\n", "We will start with a regression problem and we will reuse our codes on gradient descent methods from project 1." ] }, { "cell_type": "markdown", - "id": "6eb7d47f", + "id": "d8baab67", "metadata": { "editable": true }, @@ -143,11 +143,11 @@ "gradients. The functions whose gradients we need are:\n", "1. The mean-squared error (MSE) with and without the $L_1$ and $L_2$ norms (regression problems)\n", "\n", - "2. The binary cross entropy (aka log loss) for classification problems with and without $L_1$ and $L_2$ norms\n", + "2. The binary cross entropy (aka log loss) for binary classification problems with and without $L_1$ and $L_2$ norms\n", "\n", "3. The multiclass cross entropy cost/loss function (aka Softmax cross entropy or just Softmax loss function)\n", "\n", - "Set up these three cost/loss functions and their respective derivatives and explain the various terms.\n", + "Set up these three cost/loss functions and their respective derivatives and explain the various terms. In this project you will however only use the MSE and the Softmax cross entropy.\n", "\n", "We will test three activation functions for our neural network setup, these are the \n", "1. The Sigmoid (aka **logit**) function,\n", @@ -162,7 +162,7 @@ }, { "cell_type": "markdown", - "id": "2d071fd6", + "id": "87e7ed71", "metadata": { "editable": true }, @@ -189,7 +189,7 @@ }, { "cell_type": "markdown", - "id": "79aab198", + "id": "5a26b6ad", "metadata": { "editable": true }, @@ -205,7 +205,7 @@ }, { "cell_type": "markdown", - "id": "ce89599d", + "id": "096fe6c4", "metadata": { "editable": true }, @@ -217,7 +217,7 @@ }, { "cell_type": "markdown", - "id": "0f5c6fd3", + "id": "fd986596", "metadata": { "editable": true }, @@ -243,7 +243,7 @@ "Comment your results and give a critical discussion of the results\n", "obtained with the OLS code from project 1 and your own neural network\n", "code. Make an analysis of the learning rates employed to find the\n", - "optimal MSE and $R2$ scores. Test both stochastic gradient descent\n", + "optimal MSE score. Test both stochastic gradient descent\n", "with RMSprop and ADAM and plain gradient descent with different\n", "learning rates.\n", "\n", @@ -252,7 +252,7 @@ }, { "cell_type": "markdown", - "id": "a1ee77ac", + "id": "e853d4b6", "metadata": { "editable": true }, @@ -270,7 +270,7 @@ }, { "cell_type": "markdown", - "id": "cbbb5ef7", + "id": "fc2d413b", "metadata": { "editable": true }, @@ -285,7 +285,7 @@ }, { "cell_type": "markdown", - "id": "ec569b37", + "id": "e6821051", "metadata": { "editable": true }, @@ -302,7 +302,7 @@ }, { "cell_type": "markdown", - "id": "24327556", + "id": "cba72d68", "metadata": { "editable": true }, @@ -328,7 +328,7 @@ { "cell_type": "code", "execution_count": 1, - "id": "7fa0dca1", + "id": "e16fb528", "metadata": { "collapsed": false, "editable": true @@ -347,7 +347,7 @@ }, { "cell_type": "markdown", - "id": "f625eccf", + "id": "73599f42", "metadata": { "editable": true }, @@ -358,7 +358,7 @@ { "cell_type": "code", "execution_count": 2, - "id": "7614316d", + "id": "f1a639ef", "metadata": { "collapsed": false, "editable": true @@ -370,7 +370,7 @@ }, { "cell_type": "markdown", - "id": "9ebeee4a", + "id": "90fb7b41", "metadata": { "editable": true }, @@ -381,7 +381,7 @@ { "cell_type": "code", "execution_count": 3, - "id": "2ec6b15a", + "id": "424af629", "metadata": { "collapsed": false, "editable": true @@ -394,7 +394,7 @@ }, { "cell_type": "markdown", - "id": "7cad9a97", + "id": "3c006080", "metadata": { "editable": true }, @@ -407,7 +407,7 @@ }, { "cell_type": "markdown", - "id": "2981093c", + "id": "a18ddd54", "metadata": { "editable": true }, @@ -419,7 +419,7 @@ }, { "cell_type": "markdown", - "id": "7dee297d", + "id": "1a1afaf9", "metadata": { "editable": true }, @@ -442,7 +442,7 @@ { "cell_type": "code", "execution_count": 4, - "id": "ad088d25", + "id": "3c37cbaf", "metadata": { "collapsed": false, "editable": true @@ -464,7 +464,7 @@ }, { "cell_type": "markdown", - "id": "e0ded2d6", + "id": "106b9303", "metadata": { "editable": true }, @@ -480,7 +480,7 @@ }, { "cell_type": "markdown", - "id": "dc5dc11b", + "id": "55da0d7f", "metadata": { "editable": true }, @@ -496,7 +496,7 @@ }, { "cell_type": "markdown", - "id": "c6aeb5ed", + "id": "d3731e2c", "metadata": { "editable": true }, @@ -527,7 +527,7 @@ }, { "cell_type": "markdown", - "id": "a2da10ef", + "id": "6c7c5340", "metadata": { "editable": true }, diff --git a/doc/Projects/2025/Project2/ipynb/ipynb-Project2-src.tar.gz b/doc/Projects/2025/Project2/ipynb/ipynb-Project2-src.tar.gz index 3214acc008810f69c17e9bccabdc65fc23f2fbec..84cf6770e2fc79a5615c6c7e3e3ca4b953ea6862 100644 GIT binary patch literal 193 zcmV;y06za8iwFR~p6qA<1MSaC3c@fD2H>uHia9}uHia9|^nm*QoUAPd6c!AWWHq~Zol7hXx zeSoeMH${Yeo1bBZVdju+w)-rwck8W&5Rya@OqnL}oFrW88Ko)En52x-I0+dL%3?-5 zAnTp<(i_Y5cuHG0LK&gn&5dJK^