lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update project 2

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2025-10-13 08:41:59 +02:00
parent 8e604f173a
commit 6cf5b46ca0
9 changed files with 88 additions and 683 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/Projects/2025/Project2/html/Project2.html
+11 -11
View File
@@ -278,14 +278,14 @@ feed-forward neural network (FFNN) code. The exercises from week 41 and 42 (see
<p>The data sets that we propose here are (the default sets)</p>
<ul>
<li> Regression (fitting a continuous function). In this part you will need to bring back your results from project 1 and compare these with what you get from your Neural Network code to be developed here. The data sets could be
<ol type="a"></li>
<li> The simple one-dimensional function Runge function from project 1, that is \( f(x) = \frac{1}{1+25x^2} \). We recommend using a simpler function when developing your neural network code for regression problems. You should however feel free to discuss and study other functions, such as the the two-dimensional Runge function \( f(x,y)=\left[(10x - 5)^2 + (10y - 5)^2 + 1 \right]^{-1} \), or even more complicated two-dimensional functions (see the supplementary material of <a href="https://www.nature.com/articles/s41467-025-61362-4" target="_blank"><tt>https://www.nature.com/articles/s41467-025-61362-4</tt></a> for an extensive list of two-dimensional functions).</li>
</ol>
<li> Classification.
<ol type="a"></li>
<li> We will consider the multiclass classification problem given by the full MNIST data set. The one included in <b>scikit-learn</b> is reduced data. The full data set is at <a href="https://www.kaggle.com/datasets/hojjatk/mnist-dataset" target="_blank"><tt>https://www.kaggle.com/datasets/hojjatk/mnist-dataset</tt></a>.</li>
</ol>
<li> Regression (fitting a continuous function). In this part you will need to bring back your results from project 1 and compare these with what you get from your Neural Network code to be developed here. The data sets could be</li>
<ul>
<li> The simple one-dimensional function Runge function from project 1, that is \( f(x) = \frac{1}{1+25x^2} \). We recommend using a simpler function when developing your neural network code for regression problems. Feel however free to discuss and study other functions, such as the the two-dimensional Runge function \( f(x,y)=\left[(10x - 5)^2 + (10y - 5)^2 + 1 \right]^{-1} \), or even more complicated two-dimensional functions (see the supplementary material of <a href="https://www.nature.com/articles/s41467-025-61362-4" target="_blank"><tt>https://www.nature.com/articles/s41467-025-61362-4</tt></a> for an extensive list of two-dimensional functions).</li>
</ul>
<li> Classification.</li>
<ul>
<li> We will consider a multiclass classification problem given by the full MNIST data set. The full data set is at <a href="https://www.kaggle.com/datasets/hojjatk/mnist-dataset" target="_blank"><tt>https://www.kaggle.com/datasets/hojjatk/mnist-dataset</tt></a>.</li>
</ul>
</ul>
<p>We will start with a regression problem and we will reuse our codes on gradient descent methods from project 1.</p>
<h3 id="part-a-analytical-warm-up">Part a): Analytical warm-up </h3>
@@ -295,10 +295,10 @@ gradients. The functions whose gradients we need are:
</p>
<ol>
<li> The mean-squared error (MSE) with and without the \( L_1 \) and \( L_2 \) norms (regression problems)</li>
<li> The binary cross entropy (aka log loss) for classification problems with and without \( L_1 \) and \( L_2 \) norms</li>
<li> The binary cross entropy (aka log loss) for binary classification problems with and without \( L_1 \) and \( L_2 \) norms</li>
<li> The multiclass cross entropy cost/loss function (aka Softmax cross entropy or just Softmax loss function)</li>
</ol>
<p>Set up these three cost/loss functions and their respective derivatives and explain the various terms.</p>
<p>Set up these three cost/loss functions and their respective derivatives and explain the various terms. In this project you will however only use the MSE and the Softmax cross entropy.</p>
<p>We will test three activation functions for our neural network setup, these are the </p>
<ol>
@@ -365,7 +365,7 @@ and two hidden layers using \( 50 \) and \( 100 \) hidden nodes, respectively.
<p>Comment your results and give a critical discussion of the results
obtained with the OLS code from project 1 and your own neural network
code. Make an analysis of the learning rates employed to find the
optimal MSE and \( R2 \) scores. Test both stochastic gradient descent
optimal MSE score. Test both stochastic gradient descent
with RMSprop and ADAM and plain gradient descent with different
learning rates.
</p>