Part f): Cross-validation as resampling techniques, adding more complexity
-The aim here is to write your own code for another widely popular
+
The aim here is to implement another widely popular
resampling technique, the so-called cross-validation method.
Implement the \( k \)-fold cross-validation algorithm (write your own
-code) and evaluate again the MSE function resulting
-from the test folds. You can compare your own code with that from
-Scikit-Learn if needed.
+code or use the functionality of Scikit-Learn) and evaluate again the MSE function resulting
+from the test folds.
Compare the MSE you get from your cross-validation code with the one
you got from your bootstrap code. Comment your results. Try \( 5-10 \)
-folds. You can also compare your own cross-validation code with the
-one provided by Scikit-Learn.
+folds.
In addition to using the ordinary least squares method, you should include both Ridge and Lasso regression.
diff --git a/doc/Projects/2024/Project1/html/Project1.html b/doc/Projects/2024/Project1/html/Project1.html
index 42b54e81b..c8cafdff1 100644
--- a/doc/Projects/2024/Project1/html/Project1.html
+++ b/doc/Projects/2024/Project1/html/Project1.html
@@ -208,12 +208,21 @@ MathJax.Hub.Config({
-We want you to answer the three different projects by handing reports written like a standard scientific/technical report.
-The link at https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ProjectWriting/projectwriting.ipynb gives some guidance. See also the grading suggestion at https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md.
+
We want you to answer the three different projects by handing in
+reports written like a standard scientific/technical report. The link
+at
+https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ProjectWriting/projectwriting.ipynb
+gives some guidance. See also the grading suggestion at
+https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md.
-When using codes from different sources that you have not developed yourself,
-you should refer to these in the bibliography of your report, indicating wherefrom you
+
Furthermore, at
+https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/
+you can find examples of previous reports. How to write reports will
+also be discussed during the various lab sessions. Please do ask us if you are in doubt.
+
+
+When using codes and material from other sources, you should refer to these in the bibliography of your report, indicating wherefrom you for example
got the code, whether this is from the lecture notes, softwares like
Scikit-Learn, TensorFlow, PyTorch or other sources. These should
always be cited correctly. How to cite some of the libraries is often
@@ -221,25 +230,16 @@ indicated from their corresponding GitHub sites or websites, see for example how
We enocurage you to use tools like
-ChatGPT in writing the report. If you use for example ChatGPT,
+ChatGPT or similar in writing the report. If you use for example ChatGPT,
please do cite it properly and include (if possible) your questions and answers as an addition to the report. This can
-be uplodaed to for example your website, GitHub/GitLab or similar as supplemental material.
+be uploaded to for example your website, GitHub/GitLab or similar as supplemental material.
-
-On scaling, we recommend reading the following section from the scikit-learn software description, see https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html#plot-all-scaling-standard-scaler-section
Regression analysis and resampling methods
The main aim of this project is to study in more detail various
regression methods, including the Ordinary Least Squares (OLS) method.
In addition to the scientific part, in this course we want also to
-give you an experience in writing scientific reports. The format for
-the delivery of your answers is namely that of a scientific report. At
-for example
-https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md
-we detail how to write a report. Furthermore, at
-https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/
-you can find examples of previous reports. How to write reports will
-also be discussed during the various lab sessions.
+give you an experience in writing scientific reports.
A small recommendation when developing the codes here. Instead of
@@ -364,6 +364,8 @@ plt.show()
+
+
If you wish to compare your results with other on the Franke function or other popular functions tested with linear regression, see the list in Figure 1 of the article by Cook et al at https://arxiv.org/abs/2401.11694.
Part a) : Ordinary Least Square (OLS) on the Franke function
We will generate our own dataset for a function
@@ -423,6 +425,8 @@ approximately \( 2/3 \) to \( 4/5 \) of the data as training data.
You can easily reuse the solutions to your exercises from week 35 and week 36.
See also the lecture slides from week 35 and week 36.
+
+On scaling, we recommend reading the following section from the scikit-learn software description, see https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html#plot-all-scaling-standard-scaler-section.
Part b): Adding Ridge regression for the Franke function
Write your own code for the Ridge method, either using matrix
@@ -579,20 +583,18 @@ You can follow the code example in the jupyter-book at Part f): Cross-validation as resampling techniques, adding more complexity
-The aim here is to write your own code for another widely popular
+
The aim here is to implement another widely popular
resampling technique, the so-called cross-validation method.
Implement the \( k \)-fold cross-validation algorithm (write your own
-code) and evaluate again the MSE function resulting
-from the test folds. You can compare your own code with that from
-Scikit-Learn if needed.
+code or use the functionality of Scikit-Learn) and evaluate again the MSE function resulting
+from the test folds.
Compare the MSE you get from your cross-validation code with the one
you got from your bootstrap code. Comment your results. Try \( 5-10 \)
-folds. You can also compare your own cross-validation code with the
-one provided by Scikit-Learn.
+folds.
In addition to using the ordinary least squares method, you should include both Ridge and Lasso regression.
diff --git a/doc/Projects/2024/Project1/ipynb/Project1.ipynb b/doc/Projects/2024/Project1/ipynb/Project1.ipynb
index 71e7871a5..816cc7c52 100644
--- a/doc/Projects/2024/Project1/ipynb/Project1.ipynb
+++ b/doc/Projects/2024/Project1/ipynb/Project1.ipynb
@@ -2,7 +2,7 @@
"cells": [
{
"cell_type": "markdown",
- "id": "f29d1e5a",
+ "id": "6c8c59f8",
"metadata": {
"editable": true
},
@@ -14,7 +14,7 @@
},
{
"cell_type": "markdown",
- "id": "60f7a134",
+ "id": "24af4cf5",
"metadata": {
"editable": true
},
@@ -27,34 +27,40 @@
},
{
"cell_type": "markdown",
- "id": "43b931d0",
+ "id": "2971d68f",
"metadata": {
"editable": true
},
"source": [
"## Preamble: Note on writing reports, using reference material, AI and other tools\n",
"\n",
- "We want you to answer the three different projects by handing reports written like a standard scientific/technical report.\n",
- "The link at gives some guidance. See also the grading suggestion at .\n",
+ "We want you to answer the three different projects by handing in\n",
+ "reports written like a standard scientific/technical report. The link\n",
+ "at\n",
+ "\n",
+ "gives some guidance. See also the grading suggestion at\n",
+ ".\n",
"\n",
- "When using codes from different sources that you have not developed yourself,\n",
- "you should refer to these in the bibliography of your report, indicating wherefrom you\n",
+ "Furthermore, at\n",
+ "\n",
+ "you can find examples of previous reports. How to write reports will\n",
+ "also be discussed during the various lab sessions. Please do ask us if you are in doubt.\n",
+ "\n",
+ "When using codes and material from other sources, you should refer to these in the bibliography of your report, indicating wherefrom you for example\n",
"got the code, whether this is from the lecture notes, softwares like\n",
"Scikit-Learn, TensorFlow, PyTorch or other sources. These should\n",
"always be cited correctly. How to cite some of the libraries is often\n",
"indicated from their corresponding GitHub sites or websites, see for example how to cite Scikit-Learn at . \n",
"\n",
"We enocurage you to use tools like\n",
- "[ChatGPT](https://openai.com/chatgpt/) in writing the report. If you use for example ChatGPT,\n",
+ "[ChatGPT](https://openai.com/chatgpt/) or similar in writing the report. If you use for example ChatGPT,\n",
"please do cite it properly and include (if possible) your questions and answers as an addition to the report. This can\n",
- "be uplodaed to for example your website, GitHub/GitLab or similar as supplemental material.\n",
- "\n",
- "On scaling, we recommend reading the following section from the scikit-learn software description, see "
+ "be uploaded to for example your website, GitHub/GitLab or similar as supplemental material."
]
},
{
"cell_type": "markdown",
- "id": "131f84dc",
+ "id": "eba3be6a",
"metadata": {
"editable": true
},
@@ -64,14 +70,7 @@
"The main aim of this project is to study in more detail various\n",
"regression methods, including the Ordinary Least Squares (OLS) method.\n",
"In addition to the scientific part, in this course we want also to\n",
- "give you an experience in writing scientific reports. The format for\n",
- "the delivery of your answers is namely that of a scientific report. At\n",
- "for example\n",
- "\n",
- "we detail how to write a report. Furthermore, at\n",
- "\n",
- "you can find examples of previous reports. How to write reports will\n",
- "also be discussed during the various lab sessions.\n",
+ "give you an experience in writing scientific reports.\n",
"\n",
"**A small recommendation when developing the codes here**. Instead of\n",
"jumping on to the two-dimensional function described below, we\n",
@@ -90,7 +89,7 @@
},
{
"cell_type": "markdown",
- "id": "335be2ea",
+ "id": "50da25e5",
"metadata": {
"editable": true
},
@@ -112,7 +111,7 @@
},
{
"cell_type": "markdown",
- "id": "a77e527b",
+ "id": "7757fb0c",
"metadata": {
"editable": true
},
@@ -127,7 +126,7 @@
},
{
"cell_type": "markdown",
- "id": "071b756f",
+ "id": "83fbdb79",
"metadata": {
"editable": true
},
@@ -160,7 +159,7 @@
{
"cell_type": "code",
"execution_count": 1,
- "id": "eae160df",
+ "id": "39b900ad",
"metadata": {
"collapsed": false,
"editable": true
@@ -212,7 +211,17 @@
},
{
"cell_type": "markdown",
- "id": "c2cfaf54",
+ "id": "ef99fea4",
+ "metadata": {
+ "editable": true
+ },
+ "source": [
+ "If you wish to compare your results with other on the Franke function or other popular functions tested with linear regression, see the list in Figure 1 of the article by Cook et al at ."
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "id": "b1a7f51d",
"metadata": {
"editable": true
},
@@ -234,7 +243,7 @@
},
{
"cell_type": "markdown",
- "id": "a7cd4d8f",
+ "id": "d4bd58e3",
"metadata": {
"editable": true
},
@@ -247,7 +256,7 @@
},
{
"cell_type": "markdown",
- "id": "c0bbdb51",
+ "id": "32f767b4",
"metadata": {
"editable": true
},
@@ -259,7 +268,7 @@
},
{
"cell_type": "markdown",
- "id": "ddee9b4b",
+ "id": "bae0d9d6",
"metadata": {
"editable": true
},
@@ -271,7 +280,7 @@
},
{
"cell_type": "markdown",
- "id": "6f220536",
+ "id": "8f0b2ec9",
"metadata": {
"editable": true
},
@@ -281,7 +290,7 @@
},
{
"cell_type": "markdown",
- "id": "feb5938f",
+ "id": "8a455920",
"metadata": {
"editable": true
},
@@ -293,7 +302,7 @@
},
{
"cell_type": "markdown",
- "id": "8ca0b371",
+ "id": "3f7ffe3e",
"metadata": {
"editable": true
},
@@ -317,12 +326,14 @@
"approximately $2/3$ to $4/5$ of the data as training data.\n",
"\n",
"You can easily reuse the solutions to your exercises from week 35 and week 36.\n",
- "See also the lecture slides from week 35 and week 36."
+ "See also the lecture slides from week 35 and week 36.\n",
+ "\n",
+ "On scaling, we recommend reading the following section from the scikit-learn software description, see ."
]
},
{
"cell_type": "markdown",
- "id": "17494425",
+ "id": "cf4d70a4",
"metadata": {
"editable": true
},
@@ -340,7 +351,7 @@
},
{
"cell_type": "markdown",
- "id": "79411fbf",
+ "id": "8cc60702",
"metadata": {
"editable": true
},
@@ -357,7 +368,7 @@
},
{
"cell_type": "markdown",
- "id": "0de10c86",
+ "id": "bd9d1dd3",
"metadata": {
"editable": true
},
@@ -373,7 +384,7 @@
},
{
"cell_type": "markdown",
- "id": "b284b2d9",
+ "id": "03fab7b5",
"metadata": {
"editable": true
},
@@ -385,7 +396,7 @@
},
{
"cell_type": "markdown",
- "id": "3a862737",
+ "id": "009b7fb9",
"metadata": {
"editable": true
},
@@ -396,7 +407,7 @@
},
{
"cell_type": "markdown",
- "id": "8ca87494",
+ "id": "5bf0a0d5",
"metadata": {
"editable": true
},
@@ -408,7 +419,7 @@
},
{
"cell_type": "markdown",
- "id": "a253c39b",
+ "id": "52c48acb",
"metadata": {
"editable": true
},
@@ -420,7 +431,7 @@
},
{
"cell_type": "markdown",
- "id": "9c38fb19",
+ "id": "3158357a",
"metadata": {
"editable": true
},
@@ -432,7 +443,7 @@
},
{
"cell_type": "markdown",
- "id": "8d59d872",
+ "id": "021253bc",
"metadata": {
"editable": true
},
@@ -443,7 +454,7 @@
},
{
"cell_type": "markdown",
- "id": "0efd8089",
+ "id": "9e89d5fe",
"metadata": {
"editable": true
},
@@ -455,7 +466,7 @@
},
{
"cell_type": "markdown",
- "id": "1a8121b6",
+ "id": "5f79916c",
"metadata": {
"editable": true
},
@@ -468,7 +479,7 @@
},
{
"cell_type": "markdown",
- "id": "544da2b4",
+ "id": "a6e62eff",
"metadata": {
"editable": true
},
@@ -480,7 +491,7 @@
},
{
"cell_type": "markdown",
- "id": "73fd411a",
+ "id": "7e833f14",
"metadata": {
"editable": true
},
@@ -490,7 +501,7 @@
},
{
"cell_type": "markdown",
- "id": "b7160507",
+ "id": "14ef5a97",
"metadata": {
"editable": true
},
@@ -502,7 +513,7 @@
},
{
"cell_type": "markdown",
- "id": "b18ae3c3",
+ "id": "a9443b1d",
"metadata": {
"editable": true
},
@@ -513,7 +524,7 @@
},
{
"cell_type": "markdown",
- "id": "a57a5e8c",
+ "id": "ff0c2a46",
"metadata": {
"editable": true
},
@@ -546,7 +557,7 @@
},
{
"cell_type": "markdown",
- "id": "0a45f635",
+ "id": "4c8ea78a",
"metadata": {
"editable": true
},
@@ -558,7 +569,7 @@
},
{
"cell_type": "markdown",
- "id": "ac85531a",
+ "id": "1d119b3e",
"metadata": {
"editable": true
},
@@ -577,7 +588,7 @@
},
{
"cell_type": "markdown",
- "id": "69061e11",
+ "id": "b9782b21",
"metadata": {
"editable": true
},
@@ -589,7 +600,7 @@
},
{
"cell_type": "markdown",
- "id": "b0ed607f",
+ "id": "457bd0ae",
"metadata": {
"editable": true
},
@@ -603,7 +614,7 @@
},
{
"cell_type": "markdown",
- "id": "3ab15499",
+ "id": "fbc011e0",
"metadata": {
"editable": true
},
@@ -615,7 +626,7 @@
},
{
"cell_type": "markdown",
- "id": "f1c5e132",
+ "id": "5bb40600",
"metadata": {
"editable": true
},
@@ -625,7 +636,7 @@
},
{
"cell_type": "markdown",
- "id": "60f2d431",
+ "id": "e5aebe0a",
"metadata": {
"editable": true
},
@@ -637,7 +648,7 @@
},
{
"cell_type": "markdown",
- "id": "572f071d",
+ "id": "6f243211",
"metadata": {
"editable": true
},
@@ -647,7 +658,7 @@
},
{
"cell_type": "markdown",
- "id": "89eb115c",
+ "id": "850e1403",
"metadata": {
"editable": true
},
@@ -659,7 +670,7 @@
},
{
"cell_type": "markdown",
- "id": "9061bf93",
+ "id": "86066fab",
"metadata": {
"editable": true
},
@@ -678,32 +689,30 @@
},
{
"cell_type": "markdown",
- "id": "1df7ce8d",
+ "id": "aedb0de8",
"metadata": {
"editable": true
},
"source": [
"### Part f): Cross-validation as resampling techniques, adding more complexity\n",
"\n",
- "The aim here is to write your own code for another widely popular\n",
+ "The aim here is to implement another widely popular\n",
"resampling technique, the so-called cross-validation method. \n",
"\n",
"Implement the $k$-fold cross-validation algorithm (write your own\n",
- "code) and evaluate again the MSE function resulting\n",
- "from the test folds. You can compare your own code with that from\n",
- "**Scikit-Learn** if needed. \n",
+ "code or use the functionality of **Scikit-Learn**) and evaluate again the MSE function resulting\n",
+ "from the test folds. \n",
"\n",
"Compare the MSE you get from your cross-validation code with the one\n",
"you got from your **bootstrap** code. Comment your results. Try $5-10$\n",
- "folds. You can also compare your own cross-validation code with the\n",
- "one provided by **Scikit-Learn**.\n",
+ "folds. \n",
"\n",
"In addition to using the ordinary least squares method, you should include both Ridge and Lasso regression."
]
},
{
"cell_type": "markdown",
- "id": "f99f3b0c",
+ "id": "09e42708",
"metadata": {
"editable": true
},
@@ -731,7 +740,7 @@
{
"cell_type": "code",
"execution_count": 2,
- "id": "d9d2d89a",
+ "id": "a7412176",
"metadata": {
"collapsed": false,
"editable": true
@@ -743,7 +752,7 @@
},
{
"cell_type": "markdown",
- "id": "59f5b1a6",
+ "id": "2462a733",
"metadata": {
"editable": true
},
@@ -755,7 +764,7 @@
{
"cell_type": "code",
"execution_count": 3,
- "id": "ab7fd531",
+ "id": "e299ff99",
"metadata": {
"collapsed": false,
"editable": true
@@ -781,7 +790,7 @@
},
{
"cell_type": "markdown",
- "id": "271b3252",
+ "id": "58bfbdc9",
"metadata": {
"editable": true
},
@@ -806,7 +815,7 @@
},
{
"cell_type": "markdown",
- "id": "6555e3ff",
+ "id": "5c69b9d7",
"metadata": {
"editable": true
},
@@ -820,7 +829,7 @@
},
{
"cell_type": "markdown",
- "id": "8289021b",
+ "id": "a92b1a41",
"metadata": {
"editable": true
},
@@ -850,7 +859,7 @@
},
{
"cell_type": "markdown",
- "id": "4349e2d3",
+ "id": "3da35987",
"metadata": {
"editable": true
},
@@ -872,7 +881,7 @@
},
{
"cell_type": "markdown",
- "id": "274e1581",
+ "id": "c03bf204",
"metadata": {
"editable": true
},
diff --git a/doc/Projects/2024/Project1/ipynb/ipynb-Project1-src.tar.gz b/doc/Projects/2024/Project1/ipynb/ipynb-Project1-src.tar.gz
index b6c496c20..499227cd0 100644
Binary files a/doc/Projects/2024/Project1/ipynb/ipynb-Project1-src.tar.gz and b/doc/Projects/2024/Project1/ipynb/ipynb-Project1-src.tar.gz differ
diff --git a/doc/Projects/2024/Project1/pdf/Project1.p.tex b/doc/Projects/2024/Project1/pdf/Project1.p.tex
index bd8d91402..83bdbea44 100644
--- a/doc/Projects/2024/Project1/pdf/Project1.p.tex
+++ b/doc/Projects/2024/Project1/pdf/Project1.p.tex
@@ -141,36 +141,35 @@ September 2
\subsection{Preamble: Note on writing reports, using reference material, AI and other tools}
-We want you to answer the three different projects by handing reports written like a standard scientific/technical report.
-The link at \href{{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ProjectWriting/projectwriting.ipynb}}{\nolinkurl{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ProjectWriting/projectwriting.ipynb}} gives some guidance. See also the grading suggestion at \href{{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md}}{\nolinkurl{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md}}.
+We want you to answer the three different projects by handing in
+reports written like a standard scientific/technical report. The link
+at
+\href{{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ProjectWriting/projectwriting.ipynb}}{\nolinkurl{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ProjectWriting/projectwriting.ipynb}}
+gives some guidance. See also the grading suggestion at
+\href{{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md}}{\nolinkurl{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md}}.
-When using codes from different sources that you have not developed yourself,
-you should refer to these in the bibliography of your report, indicating wherefrom you
+Furthermore, at
+\href{{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/}}{\nolinkurl{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/}}
+you can find examples of previous reports. How to write reports will
+also be discussed during the various lab sessions. Please do ask us if you are in doubt.
+
+When using codes and material from other sources, you should refer to these in the bibliography of your report, indicating wherefrom you for example
got the code, whether this is from the lecture notes, softwares like
Scikit-Learn, TensorFlow, PyTorch or other sources. These should
always be cited correctly. How to cite some of the libraries is often
indicated from their corresponding GitHub sites or websites, see for example how to cite Scikit-Learn at \href{{https://scikit-learn.org/dev/about.html}}{\nolinkurl{https://scikit-learn.org/dev/about.html}}.
We enocurage you to use tools like
-\href{{https://openai.com/chatgpt/}}{ChatGPT} in writing the report. If you use for example ChatGPT,
+\href{{https://openai.com/chatgpt/}}{ChatGPT} or similar in writing the report. If you use for example ChatGPT,
please do cite it properly and include (if possible) your questions and answers as an addition to the report. This can
-be uplodaed to for example your website, GitHub/GitLab or similar as supplemental material.
-
-On scaling, we recommend reading the following section from the scikit-learn software description, see \href{{https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html#plot-all-scaling-standard-scaler-section}}{\nolinkurl{https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html\#plot-all-scaling-standard-scaler-section}}
+be uploaded to for example your website, GitHub/GitLab or similar as supplemental material.
\subsection{Regression analysis and resampling methods}
The main aim of this project is to study in more detail various
regression methods, including the Ordinary Least Squares (OLS) method.
In addition to the scientific part, in this course we want also to
-give you an experience in writing scientific reports. The format for
-the delivery of your answers is namely that of a scientific report. At
-for example
-\href{{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md}}{\nolinkurl{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md}}
-we detail how to write a report. Furthermore, at
-\href{{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/}}{\nolinkurl{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/}}
-you can find examples of previous reports. How to write reports will
-also be discussed during the various lab sessions.
+give you an experience in writing scientific reports.
\textbf{A small recommendation when developing the codes here}. Instead of
jumping on to the two-dimensional function described below, we
@@ -312,6 +311,8 @@ plt.show()
\epycod
+If you wish to compare your results with other on the Franke function or other popular functions tested with linear regression, see the list in Figure 1 of the article by Cook et al at \href{{https://arxiv.org/abs/2401.11694}}{\nolinkurl{https://arxiv.org/abs/2401.11694}}.
+
\paragraph{Part a) : Ordinary Least Square (OLS) on the Franke function.}
We will generate our own dataset for a function
$\mathrm{FrankeFunction}(x,y)$ with $x,y \in [0,1]$. The function
@@ -364,6 +365,8 @@ approximately $2/3$ to $4/5$ of the data as training data.
You can easily reuse the solutions to your exercises from week 35 and week 36.
See also the lecture slides from week 35 and week 36.
+On scaling, we recommend reading the following section from the scikit-learn software description, see \href{{https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html#plot-all-scaling-standard-scaler-section}}{\nolinkurl{https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html\#plot-all-scaling-standard-scaler-section}}.
+
\paragraph{Part b): Adding Ridge regression for the Franke function.}
Write your own code for the Ridge method, either using matrix
inversion or the singular value decomposition as done in the previous
@@ -492,18 +495,16 @@ of data points, and possibly also your training and test data using the \textbf{
You can follow the code example in the jupyter-book at \href{{https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/chapter3.html#the-bias-variance-tradeoff}}{\nolinkurl{https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/chapter3.html\#the-bias-variance-tradeoff}}.
\paragraph{Part f): Cross-validation as resampling techniques, adding more complexity.}
-The aim here is to write your own code for another widely popular
+The aim here is to implement another widely popular
resampling technique, the so-called cross-validation method.
Implement the $k$-fold cross-validation algorithm (write your own
-code) and evaluate again the MSE function resulting
-from the test folds. You can compare your own code with that from
-\textbf{Scikit-Learn} if needed.
+code or use the functionality of \textbf{Scikit-Learn}) and evaluate again the MSE function resulting
+from the test folds.
Compare the MSE you get from your cross-validation code with the one
you got from your \textbf{bootstrap} code. Comment your results. Try $5-10$
-folds. You can also compare your own cross-validation code with the
-one provided by \textbf{Scikit-Learn}.
+folds.
In addition to using the ordinary least squares method, you should include both Ridge and Lasso regression.
diff --git a/doc/Projects/2024/Project1/pdf/Project1.pdf b/doc/Projects/2024/Project1/pdf/Project1.pdf
index cb9449876..5cc4196ef 100644
Binary files a/doc/Projects/2024/Project1/pdf/Project1.pdf and b/doc/Projects/2024/Project1/pdf/Project1.pdf differ
diff --git a/doc/Projects/2024/Project1/pdf/Project1.tex b/doc/Projects/2024/Project1/pdf/Project1.tex
index eefb1ed98..487de2583 100644
--- a/doc/Projects/2024/Project1/pdf/Project1.tex
+++ b/doc/Projects/2024/Project1/pdf/Project1.tex
@@ -111,36 +111,35 @@ September 2
\subsection*{Preamble: Note on writing reports, using reference material, AI and other tools}
-We want you to answer the three different projects by handing reports written like a standard scientific/technical report.
-The link at \href{{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ProjectWriting/projectwriting.ipynb}}{\nolinkurl{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ProjectWriting/projectwriting.ipynb}} gives some guidance. See also the grading suggestion at \href{{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md}}{\nolinkurl{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md}}.
+We want you to answer the three different projects by handing in
+reports written like a standard scientific/technical report. The link
+at
+\href{{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ProjectWriting/projectwriting.ipynb}}{\nolinkurl{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ProjectWriting/projectwriting.ipynb}}
+gives some guidance. See also the grading suggestion at
+\href{{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md}}{\nolinkurl{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md}}.
-When using codes from different sources that you have not developed yourself,
-you should refer to these in the bibliography of your report, indicating wherefrom you
+Furthermore, at
+\href{{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/}}{\nolinkurl{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/}}
+you can find examples of previous reports. How to write reports will
+also be discussed during the various lab sessions. Please do ask us if you are in doubt.
+
+When using codes and material from other sources, you should refer to these in the bibliography of your report, indicating wherefrom you for example
got the code, whether this is from the lecture notes, softwares like
Scikit-Learn, TensorFlow, PyTorch or other sources. These should
always be cited correctly. How to cite some of the libraries is often
indicated from their corresponding GitHub sites or websites, see for example how to cite Scikit-Learn at \href{{https://scikit-learn.org/dev/about.html}}{\nolinkurl{https://scikit-learn.org/dev/about.html}}.
We enocurage you to use tools like
-\href{{https://openai.com/chatgpt/}}{ChatGPT} in writing the report. If you use for example ChatGPT,
+\href{{https://openai.com/chatgpt/}}{ChatGPT} or similar in writing the report. If you use for example ChatGPT,
please do cite it properly and include (if possible) your questions and answers as an addition to the report. This can
-be uplodaed to for example your website, GitHub/GitLab or similar as supplemental material.
-
-On scaling, we recommend reading the following section from the scikit-learn software description, see \href{{https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html#plot-all-scaling-standard-scaler-section}}{\nolinkurl{https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html\#plot-all-scaling-standard-scaler-section}}
+be uploaded to for example your website, GitHub/GitLab or similar as supplemental material.
\subsection*{Regression analysis and resampling methods}
The main aim of this project is to study in more detail various
regression methods, including the Ordinary Least Squares (OLS) method.
In addition to the scientific part, in this course we want also to
-give you an experience in writing scientific reports. The format for
-the delivery of your answers is namely that of a scientific report. At
-for example
-\href{{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md}}{\nolinkurl{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md}}
-we detail how to write a report. Furthermore, at
-\href{{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/}}{\nolinkurl{https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/}}
-you can find examples of previous reports. How to write reports will
-also be discussed during the various lab sessions.
+give you an experience in writing scientific reports.
\textbf{A small recommendation when developing the codes here}. Instead of
jumping on to the two-dimensional function described below, we
@@ -282,6 +281,8 @@ plt.show()
\end{verbatim}
+If you wish to compare your results with other on the Franke function or other popular functions tested with linear regression, see the list in Figure 1 of the article by Cook et al at \href{{https://arxiv.org/abs/2401.11694}}{\nolinkurl{https://arxiv.org/abs/2401.11694}}.
+
\paragraph{Part a) : Ordinary Least Square (OLS) on the Franke function.}
We will generate our own dataset for a function
$\mathrm{FrankeFunction}(x,y)$ with $x,y \in [0,1]$. The function
@@ -334,6 +335,8 @@ approximately $2/3$ to $4/5$ of the data as training data.
You can easily reuse the solutions to your exercises from week 35 and week 36.
See also the lecture slides from week 35 and week 36.
+On scaling, we recommend reading the following section from the scikit-learn software description, see \href{{https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html#plot-all-scaling-standard-scaler-section}}{\nolinkurl{https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html\#plot-all-scaling-standard-scaler-section}}.
+
\paragraph{Part b): Adding Ridge regression for the Franke function.}
Write your own code for the Ridge method, either using matrix
inversion or the singular value decomposition as done in the previous
@@ -462,18 +465,16 @@ of data points, and possibly also your training and test data using the \textbf{
You can follow the code example in the jupyter-book at \href{{https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/chapter3.html#the-bias-variance-tradeoff}}{\nolinkurl{https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/chapter3.html\#the-bias-variance-tradeoff}}.
\paragraph{Part f): Cross-validation as resampling techniques, adding more complexity.}
-The aim here is to write your own code for another widely popular
+The aim here is to implement another widely popular
resampling technique, the so-called cross-validation method.
Implement the $k$-fold cross-validation algorithm (write your own
-code) and evaluate again the MSE function resulting
-from the test folds. You can compare your own code with that from
-\textbf{Scikit-Learn} if needed.
+code or use the functionality of \textbf{Scikit-Learn}) and evaluate again the MSE function resulting
+from the test folds.
Compare the MSE you get from your cross-validation code with the one
you got from your \textbf{bootstrap} code. Comment your results. Try $5-10$
-folds. You can also compare your own cross-validation code with the
-one provided by \textbf{Scikit-Learn}.
+folds.
In addition to using the ordinary least squares method, you should include both Ridge and Lasso regression.
diff --git a/doc/src/Projects/2024/Project1/Project1.do.txt b/doc/src/Projects/2024/Project1/Project1.do.txt
index 562d72ec9..d4f124fb5 100644
--- a/doc/src/Projects/2024/Project1/Project1.do.txt
+++ b/doc/src/Projects/2024/Project1/Project1.do.txt
@@ -3,28 +3,34 @@ AUTHOR: "Data Analysis and Machine Learning FYS-STK3155/FYS4155":"http://www.uio
DATE: September 2
-
-
-
===== Preamble: Note on writing reports, using reference material, AI and other tools =====
-We want you to answer the three different projects by handing reports written like a standard scientific/technical report.
-The link at URL:"https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ProjectWriting/projectwriting.ipynb" gives some guidance. See also the grading suggestion at URL:"https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md".
+We want you to answer the three different projects by handing in
+reports written like a standard scientific/technical report. The link
+at
+URL:"https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ProjectWriting/projectwriting.ipynb"
+gives some guidance. See also the grading suggestion at
+URL:"https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md".
-When using codes from different sources that you have not developed yourself,
-you should refer to these in the bibliography of your report, indicating wherefrom you
+Furthermore, at
+URL:"https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/"
+you can find examples of previous reports. How to write reports will
+also be discussed during the various lab sessions. Please do ask us if you are in doubt.
+
+
+
+When using codes and material from other sources, you should refer to these in the bibliography of your report, indicating wherefrom you for example
got the code, whether this is from the lecture notes, softwares like
Scikit-Learn, TensorFlow, PyTorch or other sources. These should
always be cited correctly. How to cite some of the libraries is often
indicated from their corresponding GitHub sites or websites, see for example how to cite Scikit-Learn at URL:"https://scikit-learn.org/dev/about.html".
We enocurage you to use tools like
-"ChatGPT":"https://openai.com/chatgpt/" in writing the report. If you use for example ChatGPT,
+"ChatGPT":"https://openai.com/chatgpt/" or similar in writing the report. If you use for example ChatGPT,
please do cite it properly and include (if possible) your questions and answers as an addition to the report. This can
-be uplodaed to for example your website, GitHub/GitLab or similar as supplemental material.
+be uploaded to for example your website, GitHub/GitLab or similar as supplemental material.
-On scaling, we recommend reading the following section from the scikit-learn software description, see URL:"https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html#plot-all-scaling-standard-scaler-section"
@@ -36,14 +42,8 @@ On scaling, we recommend reading the following section from the scikit-learn sof
The main aim of this project is to study in more detail various
regression methods, including the Ordinary Least Squares (OLS) method.
In addition to the scientific part, in this course we want also to
-give you an experience in writing scientific reports. The format for
-the delivery of your answers is namely that of a scientific report. At
-for example
-URL:"https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md"
-we detail how to write a report. Furthermore, at
-URL:"https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/"
-you can find examples of previous reports. How to write reports will
-also be discussed during the various lab sessions.
+give you an experience in writing scientific reports.
+
_A small recommendation when developing the codes here_. Instead of
@@ -151,6 +151,8 @@ plt.show()
!ec
+If you wish to compare your results with other on the Franke function or other popular functions tested with linear regression, see the list in Figure 1 of the article by Cook et al at URL:"https://arxiv.org/abs/2401.11694".
+
=== Part a) : Ordinary Least Square (OLS) on the Franke function ===
We will generate our own dataset for a function
@@ -213,6 +215,9 @@ approximately $2/3$ to $4/5$ of the data as training data.
You can easily reuse the solutions to your exercises from week 35 and week 36.
See also the lecture slides from week 35 and week 36.
+On scaling, we recommend reading the following section from the scikit-learn software description, see URL:"https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html#plot-all-scaling-standard-scaler-section".
+
+
=== Part b): Adding Ridge regression for the Franke function ===
@@ -381,18 +386,16 @@ You can follow the code example in the jupyter-book at URL:"https://compphysics.
=== Part f): Cross-validation as resampling techniques, adding more complexity ===
-The aim here is to write your own code for another widely popular
+The aim here is to implement another widely popular
resampling technique, the so-called cross-validation method.
Implement the $k$-fold cross-validation algorithm (write your own
-code) and evaluate again the MSE function resulting
-from the test folds. You can compare your own code with that from
-_Scikit-Learn_ if needed.
+code or use the functionality of _Scikit-Learn_) and evaluate again the MSE function resulting
+from the test folds.
Compare the MSE you get from your cross-validation code with the one
you got from your _bootstrap_ code. Comment your results. Try $5-10$
-folds. You can also compare your own cross-validation code with the
-one provided by _Scikit-Learn_.
+folds.
In addition to using the ordinary least squares method, you should include both Ridge and Lasso regression.