lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

setting up project 2

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2021-10-12 09:12:19 +02:00
parent ecac466b5f
commit 23616739e0
10 changed files with 2470 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/src/Projects/2021/Project2/Project2.do.txt
+18 -22
View File
@@ -1,4 +1,4 @@
TITLE: Project 2 on Machine Learning, deadline November 13 (Midnight)
TITLE: Project 2 on Machine Learning, deadline November 15 (Midnight)
AUTHOR: "Data Analysis and Machine Learning FYS-STK3155/FYS4155":"http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html" {copyright, 1999-present|CC BY-NC} at Department of Physics, University of Oslo, Norway
DATE: today
@@ -10,25 +10,27 @@ regression problems by developing our own feed-forward neural network (FFNN) cod
in project 1. We will also include logistic regression for classification
problems and write our own FFNN code for studying
both regression and classification problems. The codes developed in
project 1, including bootstrap and/or cross-validation as well as the
project 1, including bootstrap _and/or_ cross-validation as well as the
computation of the mean-squared error and/or the $R2$ or the accuracy score (classification problems) functions can
also be utilized in the present analysis.
_Important Note_: This project as well as project 3 have to be written as a scientific report. The instructions on how to do this and how we grade are available at URL:"https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md". We will discuss this format during the various lab sessions. Please do spend some time to read our guidelines.
The data sets that we propose here are (the default sets)
* Regression (fitting a continuous function). In this part you will need to bring back your results from project 1 and compare these with what you get from your Neural Network code to be developed here. The data sets could be
o Either the Franke function or the terrain data from project 1, or data sets your propose.
* Classification. Here you will also need to develop a Logistic regression code that you will use to compare with the Neural Network code. The data set we propose are the so-called "MNIST":"https://en.wikipedia.org/wiki/MNIST_database" data set of images representing hand-written numbers from zero to nine. These are discussed intensively in the lecture notes on neural networks, see for example the slides from "week 41":"https://compphysics.github.io/MachineLearning/doc/pub/week41/html/week41.html"
* Classification. Here you will also need to develop a Logistic regression code that you will use to compare with the Neural Network code. The data set we propose are the so-called "Wisconsin Breat Cancer Data":"https://www.kaggle.com/uciml/breast-cancer-wisconsin-data" data set of images representing various features of tumors. These are discussed intensively in the lecture notes on neural networks, see for example the slides from "week 40":"https://compphysics.github.io/MachineLearning/doc/pub/week41/html/week40.html". A longer explanation with links to the scientific literature can be found at the "Machine Learning repository of the University of California at Irvine":"https://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+%28Diagnostic%29". Feel free to consult this site and the pertinent literature.
You can find more information about this at the "Scikit-Learn site":"https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_breast_cancer.html" or at the "University of California at Irvine":"https://archive.ics.uci.edu/ml/datasets/breast+cancer+wisconsin+(original)".
However, if you would like to study other data sets, feel free to
propose other sets. What we listed here are mere suggestions from our
propose other sets. What we list here are mere suggestions from our
side. If you opt for another data set, consider using a set which
has been studied in the scientific literature. This makes it easier
for you to compare and analyze your results. Comparing with existing results from the scientific literature is also an essential
element of the scientific discussion.
element of the scientific discussion. The University of California at Irvine with its Machine Learning repository at URL:"https://archive.ics.uci.edu/ml/index.php" is an excellent site to look up for examples and inspiration. "Kaggle.com":"https://www.kaggle.com/" is an equally interesting site. Feel free to explore these sites.
In particular, when developing your own Neural Network and Logistic Regression codes for classification problems, the so-called Wisconsin Cancer data (which is a binary problem, benign or malignant tumors) may be studied. You can find more information about this at the "Scikit-Learn site":"https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_breast_cancer.html" or at the "University of California at Irvine":"https://archive.ics.uci.edu/ml/datasets/breast+cancer+wisconsin+(original)".
We will start with a regression problem and we will reuse our codes from project 1 starting with writing our own Stochastic Gradient Descent (SGD) code.
@@ -38,7 +40,7 @@ In order to get started, we will now replace in our standard ordinary
least squares (OLS) and Ridge regression codes (from project 1) the matrix inversion
algorithm with our own SGD code. You can choose whether you want to
add the momentum SGD optionality or other SGD variants such as RMSprop
or ADAgrad. The lecture notes from "week 40 contain more
or ADAgrad or ADAM. The lecture notes from "week 40 contain more
details":"https://compphysics.github.io/MachineLearning/doc/pub/week40/html/week40.html"
Perform an analysis of the results for OLS and Ridge regression as
@@ -95,11 +97,11 @@ activation function for the output layer.
Here we will change the cost function for our neural network code
developed in parts b) and c) in order to perform a classification analysis.
We will here study the MNIST data set of hand-written numbers as
discussed in the lecture notes from "week
41":"https://compphysics.github.io/MachineLearning/doc/pub/week41/html/week41.html". Use
the _Softmax_ function as activation function. Your code should
however also be able to use a binary activation function as well.
We will here study the Wisconsin Breast Cancer data set. This is a typical binary classification problem with just one single output, either True or Fale, $0$ or $1$ etc.
You find more information about this at the "Scikit-Learn
site":"https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_breast_cancer.html" or at the "University of California
at Irvine":"https://archive.ics.uci.edu/ml/datasets/breast+cancer+wisconsin+(original)".
To measure the performance of our classification problem we use the
so-called *accuracy* score. The accuracy is as you would expect just
@@ -121,14 +123,7 @@ Discuss your results and give a critical analysis of the various parameters, inc
As stated in the introduction, it can also be useful to study other
datasets. In particular, the so-called Wisconsin Cancer
data (which is a binary problem, benign or malignant tumors) may be
studied. You find more information about this at the "Scikit-Learn
site":"https://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_breast_cancer.html" or at the "University of California
at Irvine":"https://archive.ics.uci.edu/ml/datasets/breast+cancer+wisconsin+(original)".
datasets.
Again, we strongly recommend that you compare your own neural Network
code for classification and pertinent results against a similar code using _Scikit-Learn_ or _tensorflow/keras_ or _pytorch_.
@@ -145,10 +140,11 @@ classification results with the results we can obtain with another
method.
Define your cost function and the design matrix before you start writing your code.
Write thereafter a Logistic regression code using your SGD algorithm. Study the results as functions of the chosen learning rates.
Write thereafter a Logistic regression code using your SGD algorithm. You can also use standard gradient descent in this case, with a learning rate as hyper-parameter.
Study the results as functions of the chosen learning rates.
Add also an $l_2$ regularization parameter $\lambda$. Compare your results with those from your FFNN code as well as those obtained using _Scikit-Learn_'s logistic regression functionality.
The weblink here URL:"https://medium.com/ai-in-plain-english/comparison-between-logistic-regression-and-neural-networks-in-classifying-digits-dc5e85cd93c3"compares logistic regression and FFNN using the MNIST data set. You may find several useful hints and ideas from this article.
The weblink here URL:"https://medium.com/ai-in-plain-english/comparison-between-logistic-regression-and-neural-networks-in-classifying-digits-dc5e85cd93c3"compares logistic regression and FFNN using the so-called MNIST data set. You may find several useful hints and ideas from this article.
=== Part f) Critical evaluation of the various algorithms ===
doc/src/Projects/2021/Project2/make.sh
+1 -1
View File
@@ -51,7 +51,7 @@ mv -f $name.pdf ${name}.pdf
cp $name.tex ${name}.tex
# Publish
dest=../../../../Projects/2020
dest=../../../../Projects/2021
if [ ! -d $dest/$name ]; then
mkdir $dest/$name
mkdir $dest/$name/pdf