lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

added p1

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2024-09-03 05:11:10 +02:00
parent 42b46e519c
commit b01d1a6706
14 changed files with 298 additions and 271 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/LectureNotes/Project1.ipynb
+83 -74
View File
@@ -2,7 +2,7 @@
"cells": [
{
"cell_type": "markdown",
"id": "f29d1e5a",
"id": "6c8c59f8",
"metadata": {
"editable": true
},
@@ -14,7 +14,7 @@
},
{
"cell_type": "markdown",
"id": "60f7a134",
"id": "24af4cf5",
"metadata": {
"editable": true
},
@@ -27,34 +27,40 @@
},
{
"cell_type": "markdown",
"id": "43b931d0",
"id": "2971d68f",
"metadata": {
"editable": true
},
"source": [
"## Preamble: Note on writing reports, using reference material, AI and other tools\n",
"\n",
"We want you to answer the three different projects by handing reports written like a standard scientific/technical report.\n",
"The link at <https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ProjectWriting/projectwriting.ipynb> gives some guidance. See also the grading suggestion at <https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md>.\n",
"We want you to answer the three different projects by handing in\n",
"reports written like a standard scientific/technical report. The link\n",
"at\n",
"<https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ProjectWriting/projectwriting.ipynb>\n",
"gives some guidance. See also the grading suggestion at\n",
"<https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md>.\n",
"\n",
"When using codes from different sources that you have not developed yourself,\n",
"you should refer to these in the bibliography of your report, indicating wherefrom you\n",
"Furthermore, at\n",
"<https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/>\n",
"you can find examples of previous reports. How to write reports will\n",
"also be discussed during the various lab sessions. Please do ask us if you are in doubt.\n",
"\n",
"When using codes and material from other sources, you should refer to these in the bibliography of your report, indicating wherefrom you for example\n",
"got the code, whether this is from the lecture notes, softwares like\n",
"Scikit-Learn, TensorFlow, PyTorch or other sources. These should\n",
"always be cited correctly. How to cite some of the libraries is often\n",
"indicated from their corresponding GitHub sites or websites, see for example how to cite Scikit-Learn at <https://scikit-learn.org/dev/about.html>. \n",
"\n",
"We enocurage you to use tools like\n",
"[ChatGPT](https://openai.com/chatgpt/) in writing the report. If you use for example ChatGPT,\n",
"[ChatGPT](https://openai.com/chatgpt/) or similar in writing the report. If you use for example ChatGPT,\n",
"please do cite it properly and include (if possible) your questions and answers as an addition to the report. This can\n",
"be uplodaed to for example your website, GitHub/GitLab or similar as supplemental material.\n",
"\n",
"On scaling, we recommend reading the following section from the scikit-learn software description, see <https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html#plot-all-scaling-standard-scaler-section>"
"be uploaded to for example your website, GitHub/GitLab or similar as supplemental material."
]
},
{
"cell_type": "markdown",
"id": "131f84dc",
"id": "eba3be6a",
"metadata": {
"editable": true
},
@@ -64,14 +70,7 @@
"The main aim of this project is to study in more detail various\n",
"regression methods, including the Ordinary Least Squares (OLS) method.\n",
"In addition to the scientific part, in this course we want also to\n",
"give you an experience in writing scientific reports. The format for\n",
"the delivery of your answers is namely that of a scientific report. At\n",
"for example\n",
"<https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md>\n",
"we detail how to write a report. Furthermore, at\n",
"<https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/>\n",
"you can find examples of previous reports. How to write reports will\n",
"also be discussed during the various lab sessions.\n",
"give you an experience in writing scientific reports.\n",
"\n",
"**A small recommendation when developing the codes here**. Instead of\n",
"jumping on to the two-dimensional function described below, we\n",
@@ -90,7 +89,7 @@
},
{
"cell_type": "markdown",
"id": "335be2ea",
"id": "50da25e5",
"metadata": {
"editable": true
},
@@ -112,7 +111,7 @@
},
{
"cell_type": "markdown",
"id": "a77e527b",
"id": "7757fb0c",
"metadata": {
"editable": true
},
@@ -127,7 +126,7 @@
},
{
"cell_type": "markdown",
"id": "071b756f",
"id": "83fbdb79",
"metadata": {
"editable": true
},
@@ -160,7 +159,7 @@
{
"cell_type": "code",
"execution_count": 1,
"id": "eae160df",
"id": "39b900ad",
"metadata": {
"collapsed": false,
"editable": true
@@ -212,7 +211,17 @@
},
{
"cell_type": "markdown",
"id": "c2cfaf54",
"id": "ef99fea4",
"metadata": {
"editable": true
},
"source": [
"If you wish to compare your results with other on the Franke function or other popular functions tested with linear regression, see the list in Figure 1 of the article by Cook et al at <https://arxiv.org/abs/2401.11694>."
]
},
{
"cell_type": "markdown",
"id": "b1a7f51d",
"metadata": {
"editable": true
},
@@ -234,7 +243,7 @@
},
{
"cell_type": "markdown",
"id": "a7cd4d8f",
"id": "d4bd58e3",
"metadata": {
"editable": true
},
@@ -247,7 +256,7 @@
},
{
"cell_type": "markdown",
"id": "c0bbdb51",
"id": "32f767b4",
"metadata": {
"editable": true
},
@@ -259,7 +268,7 @@
},
{
"cell_type": "markdown",
"id": "ddee9b4b",
"id": "bae0d9d6",
"metadata": {
"editable": true
},
@@ -271,7 +280,7 @@
},
{
"cell_type": "markdown",
"id": "6f220536",
"id": "8f0b2ec9",
"metadata": {
"editable": true
},
@@ -281,7 +290,7 @@
},
{
"cell_type": "markdown",
"id": "feb5938f",
"id": "8a455920",
"metadata": {
"editable": true
},
@@ -293,7 +302,7 @@
},
{
"cell_type": "markdown",
"id": "8ca0b371",
"id": "3f7ffe3e",
"metadata": {
"editable": true
},
@@ -317,12 +326,14 @@
"approximately $2/3$ to $4/5$ of the data as training data.\n",
"\n",
"You can easily reuse the solutions to your exercises from week 35 and week 36.\n",
"See also the lecture slides from week 35 and week 36."
"See also the lecture slides from week 35 and week 36.\n",
"\n",
"On scaling, we recommend reading the following section from the scikit-learn software description, see <https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html#plot-all-scaling-standard-scaler-section>."
]
},
{
"cell_type": "markdown",
"id": "17494425",
"id": "cf4d70a4",
"metadata": {
"editable": true
},
@@ -340,7 +351,7 @@
},
{
"cell_type": "markdown",
"id": "79411fbf",
"id": "8cc60702",
"metadata": {
"editable": true
},
@@ -357,7 +368,7 @@
},
{
"cell_type": "markdown",
"id": "0de10c86",
"id": "bd9d1dd3",
"metadata": {
"editable": true
},
@@ -373,7 +384,7 @@
},
{
"cell_type": "markdown",
"id": "b284b2d9",
"id": "03fab7b5",
"metadata": {
"editable": true
},
@@ -385,7 +396,7 @@
},
{
"cell_type": "markdown",
"id": "3a862737",
"id": "009b7fb9",
"metadata": {
"editable": true
},
@@ -396,7 +407,7 @@
},
{
"cell_type": "markdown",
"id": "8ca87494",
"id": "5bf0a0d5",
"metadata": {
"editable": true
},
@@ -408,7 +419,7 @@
},
{
"cell_type": "markdown",
"id": "a253c39b",
"id": "52c48acb",
"metadata": {
"editable": true
},
@@ -420,7 +431,7 @@
},
{
"cell_type": "markdown",
"id": "9c38fb19",
"id": "3158357a",
"metadata": {
"editable": true
},
@@ -432,7 +443,7 @@
},
{
"cell_type": "markdown",
"id": "8d59d872",
"id": "021253bc",
"metadata": {
"editable": true
},
@@ -443,7 +454,7 @@
},
{
"cell_type": "markdown",
"id": "0efd8089",
"id": "9e89d5fe",
"metadata": {
"editable": true
},
@@ -455,7 +466,7 @@
},
{
"cell_type": "markdown",
"id": "1a8121b6",
"id": "5f79916c",
"metadata": {
"editable": true
},
@@ -468,7 +479,7 @@
},
{
"cell_type": "markdown",
"id": "544da2b4",
"id": "a6e62eff",
"metadata": {
"editable": true
},
@@ -480,7 +491,7 @@
},
{
"cell_type": "markdown",
"id": "73fd411a",
"id": "7e833f14",
"metadata": {
"editable": true
},
@@ -490,7 +501,7 @@
},
{
"cell_type": "markdown",
"id": "b7160507",
"id": "14ef5a97",
"metadata": {
"editable": true
},
@@ -502,7 +513,7 @@
},
{
"cell_type": "markdown",
"id": "b18ae3c3",
"id": "a9443b1d",
"metadata": {
"editable": true
},
@@ -513,7 +524,7 @@
},
{
"cell_type": "markdown",
"id": "a57a5e8c",
"id": "ff0c2a46",
"metadata": {
"editable": true
},
@@ -546,7 +557,7 @@
},
{
"cell_type": "markdown",
"id": "0a45f635",
"id": "4c8ea78a",
"metadata": {
"editable": true
},
@@ -558,7 +569,7 @@
},
{
"cell_type": "markdown",
"id": "ac85531a",
"id": "1d119b3e",
"metadata": {
"editable": true
},
@@ -577,7 +588,7 @@
},
{
"cell_type": "markdown",
"id": "69061e11",
"id": "b9782b21",
"metadata": {
"editable": true
},
@@ -589,7 +600,7 @@
},
{
"cell_type": "markdown",
"id": "b0ed607f",
"id": "457bd0ae",
"metadata": {
"editable": true
},
@@ -603,7 +614,7 @@
},
{
"cell_type": "markdown",
"id": "3ab15499",
"id": "fbc011e0",
"metadata": {
"editable": true
},
@@ -615,7 +626,7 @@
},
{
"cell_type": "markdown",
"id": "f1c5e132",
"id": "5bb40600",
"metadata": {
"editable": true
},
@@ -625,7 +636,7 @@
},
{
"cell_type": "markdown",
"id": "60f2d431",
"id": "e5aebe0a",
"metadata": {
"editable": true
},
@@ -637,7 +648,7 @@
},
{
"cell_type": "markdown",
"id": "572f071d",
"id": "6f243211",
"metadata": {
"editable": true
},
@@ -647,7 +658,7 @@
},
{
"cell_type": "markdown",
"id": "89eb115c",
"id": "850e1403",
"metadata": {
"editable": true
},
@@ -659,7 +670,7 @@
},
{
"cell_type": "markdown",
"id": "9061bf93",
"id": "86066fab",
"metadata": {
"editable": true
},
@@ -678,32 +689,30 @@
},
{
"cell_type": "markdown",
"id": "1df7ce8d",
"id": "aedb0de8",
"metadata": {
"editable": true
},
"source": [
"### Part f): Cross-validation as resampling techniques, adding more complexity\n",
"\n",
"The aim here is to write your own code for another widely popular\n",
"The aim here is to implement another widely popular\n",
"resampling technique, the so-called cross-validation method. \n",
"\n",
"Implement the $k$-fold cross-validation algorithm (write your own\n",
"code) and evaluate again the MSE function resulting\n",
"from the test folds. You can compare your own code with that from\n",
"**Scikit-Learn** if needed. \n",
"code or use the functionality of **Scikit-Learn**) and evaluate again the MSE function resulting\n",
"from the test folds. \n",
"\n",
"Compare the MSE you get from your cross-validation code with the one\n",
"you got from your **bootstrap** code. Comment your results. Try $5-10$\n",
"folds. You can also compare your own cross-validation code with the\n",
"one provided by **Scikit-Learn**.\n",
"folds. \n",
"\n",
"In addition to using the ordinary least squares method, you should include both Ridge and Lasso regression."
]
},
{
"cell_type": "markdown",
"id": "f99f3b0c",
"id": "09e42708",
"metadata": {
"editable": true
},
@@ -731,7 +740,7 @@
{
"cell_type": "code",
"execution_count": 2,
"id": "d9d2d89a",
"id": "a7412176",
"metadata": {
"collapsed": false,
"editable": true
@@ -743,7 +752,7 @@
},
{
"cell_type": "markdown",
"id": "59f5b1a6",
"id": "2462a733",
"metadata": {
"editable": true
},
@@ -755,7 +764,7 @@
{
"cell_type": "code",
"execution_count": 3,
"id": "ab7fd531",
"id": "e299ff99",
"metadata": {
"collapsed": false,
"editable": true
@@ -781,7 +790,7 @@
},
{
"cell_type": "markdown",
"id": "271b3252",
"id": "58bfbdc9",
"metadata": {
"editable": true
},
@@ -806,7 +815,7 @@
},
{
"cell_type": "markdown",
"id": "6555e3ff",
"id": "5c69b9d7",
"metadata": {
"editable": true
},
@@ -820,7 +829,7 @@
},
{
"cell_type": "markdown",
"id": "8289021b",
"id": "a92b1a41",
"metadata": {
"editable": true
},
@@ -850,7 +859,7 @@
},
{
"cell_type": "markdown",
"id": "4349e2d3",
"id": "3da35987",
"metadata": {
"editable": true
},
@@ -872,7 +881,7 @@
},
{
"cell_type": "markdown",
"id": "274e1581",
"id": "c03bf204",
"metadata": {
"editable": true
},
doc/LectureNotes/_build/.doctrees/environment.pickle
BIN
View File
Binary file not shown.
doc/LectureNotes/_build/.doctrees/intro.doctree
BIN
View File
Binary file not shown.
doc/LectureNotes/_build/.doctrees/project1.doctree
BIN
View File
Binary file not shown.
doc/LectureNotes/_build/html/_sources/project1.ipynb
+83 -74
View File
@@ -2,7 +2,7 @@
"cells": [
{
"cell_type": "markdown",
"id": "f29d1e5a",
"id": "6c8c59f8",
"metadata": {
"editable": true
},
@@ -14,7 +14,7 @@
},
{
"cell_type": "markdown",
"id": "60f7a134",
"id": "24af4cf5",
"metadata": {
"editable": true
},
@@ -27,34 +27,40 @@
},
{
"cell_type": "markdown",
"id": "43b931d0",
"id": "2971d68f",
"metadata": {
"editable": true
},
"source": [
"## Preamble: Note on writing reports, using reference material, AI and other tools\n",
"\n",
"We want you to answer the three different projects by handing reports written like a standard scientific/technical report.\n",
"The link at <https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ProjectWriting/projectwriting.ipynb> gives some guidance. See also the grading suggestion at <https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md>.\n",
"We want you to answer the three different projects by handing in\n",
"reports written like a standard scientific/technical report. The link\n",
"at\n",
"<https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ProjectWriting/projectwriting.ipynb>\n",
"gives some guidance. See also the grading suggestion at\n",
"<https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md>.\n",
"\n",
"When using codes from different sources that you have not developed yourself,\n",
"you should refer to these in the bibliography of your report, indicating wherefrom you\n",
"Furthermore, at\n",
"<https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/>\n",
"you can find examples of previous reports. How to write reports will\n",
"also be discussed during the various lab sessions. Please do ask us if you are in doubt.\n",
"\n",
"When using codes and material from other sources, you should refer to these in the bibliography of your report, indicating wherefrom you for example\n",
"got the code, whether this is from the lecture notes, softwares like\n",
"Scikit-Learn, TensorFlow, PyTorch or other sources. These should\n",
"always be cited correctly. How to cite some of the libraries is often\n",
"indicated from their corresponding GitHub sites or websites, see for example how to cite Scikit-Learn at <https://scikit-learn.org/dev/about.html>. \n",
"\n",
"We enocurage you to use tools like\n",
"[ChatGPT](https://openai.com/chatgpt/) in writing the report. If you use for example ChatGPT,\n",
"[ChatGPT](https://openai.com/chatgpt/) or similar in writing the report. If you use for example ChatGPT,\n",
"please do cite it properly and include (if possible) your questions and answers as an addition to the report. This can\n",
"be uplodaed to for example your website, GitHub/GitLab or similar as supplemental material.\n",
"\n",
"On scaling, we recommend reading the following section from the scikit-learn software description, see <https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html#plot-all-scaling-standard-scaler-section>"
"be uploaded to for example your website, GitHub/GitLab or similar as supplemental material."
]
},
{
"cell_type": "markdown",
"id": "131f84dc",
"id": "eba3be6a",
"metadata": {
"editable": true
},
@@ -64,14 +70,7 @@
"The main aim of this project is to study in more detail various\n",
"regression methods, including the Ordinary Least Squares (OLS) method.\n",
"In addition to the scientific part, in this course we want also to\n",
"give you an experience in writing scientific reports. The format for\n",
"the delivery of your answers is namely that of a scientific report. At\n",
"for example\n",
"<https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md>\n",
"we detail how to write a report. Furthermore, at\n",
"<https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/>\n",
"you can find examples of previous reports. How to write reports will\n",
"also be discussed during the various lab sessions.\n",
"give you an experience in writing scientific reports.\n",
"\n",
"**A small recommendation when developing the codes here**. Instead of\n",
"jumping on to the two-dimensional function described below, we\n",
@@ -90,7 +89,7 @@
},
{
"cell_type": "markdown",
"id": "335be2ea",
"id": "50da25e5",
"metadata": {
"editable": true
},
@@ -112,7 +111,7 @@
},
{
"cell_type": "markdown",
"id": "a77e527b",
"id": "7757fb0c",
"metadata": {
"editable": true
},
@@ -127,7 +126,7 @@
},
{
"cell_type": "markdown",
"id": "071b756f",
"id": "83fbdb79",
"metadata": {
"editable": true
},
@@ -160,7 +159,7 @@
{
"cell_type": "code",
"execution_count": 1,
"id": "eae160df",
"id": "39b900ad",
"metadata": {
"collapsed": false,
"editable": true
@@ -212,7 +211,17 @@
},
{
"cell_type": "markdown",
"id": "c2cfaf54",
"id": "ef99fea4",
"metadata": {
"editable": true
},
"source": [
"If you wish to compare your results with other on the Franke function or other popular functions tested with linear regression, see the list in Figure 1 of the article by Cook et al at <https://arxiv.org/abs/2401.11694>."
]
},
{
"cell_type": "markdown",
"id": "b1a7f51d",
"metadata": {
"editable": true
},
@@ -234,7 +243,7 @@
},
{
"cell_type": "markdown",
"id": "a7cd4d8f",
"id": "d4bd58e3",
"metadata": {
"editable": true
},
@@ -247,7 +256,7 @@
},
{
"cell_type": "markdown",
"id": "c0bbdb51",
"id": "32f767b4",
"metadata": {
"editable": true
},
@@ -259,7 +268,7 @@
},
{
"cell_type": "markdown",
"id": "ddee9b4b",
"id": "bae0d9d6",
"metadata": {
"editable": true
},
@@ -271,7 +280,7 @@
},
{
"cell_type": "markdown",
"id": "6f220536",
"id": "8f0b2ec9",
"metadata": {
"editable": true
},
@@ -281,7 +290,7 @@
},
{
"cell_type": "markdown",
"id": "feb5938f",
"id": "8a455920",
"metadata": {
"editable": true
},
@@ -293,7 +302,7 @@
},
{
"cell_type": "markdown",
"id": "8ca0b371",
"id": "3f7ffe3e",
"metadata": {
"editable": true
},
@@ -317,12 +326,14 @@
"approximately $2/3$ to $4/5$ of the data as training data.\n",
"\n",
"You can easily reuse the solutions to your exercises from week 35 and week 36.\n",
"See also the lecture slides from week 35 and week 36."
"See also the lecture slides from week 35 and week 36.\n",
"\n",
"On scaling, we recommend reading the following section from the scikit-learn software description, see <https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html#plot-all-scaling-standard-scaler-section>."
]
},
{
"cell_type": "markdown",
"id": "17494425",
"id": "cf4d70a4",
"metadata": {
"editable": true
},
@@ -340,7 +351,7 @@
},
{
"cell_type": "markdown",
"id": "79411fbf",
"id": "8cc60702",
"metadata": {
"editable": true
},
@@ -357,7 +368,7 @@
},
{
"cell_type": "markdown",
"id": "0de10c86",
"id": "bd9d1dd3",
"metadata": {
"editable": true
},
@@ -373,7 +384,7 @@
},
{
"cell_type": "markdown",
"id": "b284b2d9",
"id": "03fab7b5",
"metadata": {
"editable": true
},
@@ -385,7 +396,7 @@
},
{
"cell_type": "markdown",
"id": "3a862737",
"id": "009b7fb9",
"metadata": {
"editable": true
},
@@ -396,7 +407,7 @@
},
{
"cell_type": "markdown",
"id": "8ca87494",
"id": "5bf0a0d5",
"metadata": {
"editable": true
},
@@ -408,7 +419,7 @@
},
{
"cell_type": "markdown",
"id": "a253c39b",
"id": "52c48acb",
"metadata": {
"editable": true
},
@@ -420,7 +431,7 @@
},
{
"cell_type": "markdown",
"id": "9c38fb19",
"id": "3158357a",
"metadata": {
"editable": true
},
@@ -432,7 +443,7 @@
},
{
"cell_type": "markdown",
"id": "8d59d872",
"id": "021253bc",
"metadata": {
"editable": true
},
@@ -443,7 +454,7 @@
},
{
"cell_type": "markdown",
"id": "0efd8089",
"id": "9e89d5fe",
"metadata": {
"editable": true
},
@@ -455,7 +466,7 @@
},
{
"cell_type": "markdown",
"id": "1a8121b6",
"id": "5f79916c",
"metadata": {
"editable": true
},
@@ -468,7 +479,7 @@
},
{
"cell_type": "markdown",
"id": "544da2b4",
"id": "a6e62eff",
"metadata": {
"editable": true
},
@@ -480,7 +491,7 @@
},
{
"cell_type": "markdown",
"id": "73fd411a",
"id": "7e833f14",
"metadata": {
"editable": true
},
@@ -490,7 +501,7 @@
},
{
"cell_type": "markdown",
"id": "b7160507",
"id": "14ef5a97",
"metadata": {
"editable": true
},
@@ -502,7 +513,7 @@
},
{
"cell_type": "markdown",
"id": "b18ae3c3",
"id": "a9443b1d",
"metadata": {
"editable": true
},
@@ -513,7 +524,7 @@
},
{
"cell_type": "markdown",
"id": "a57a5e8c",
"id": "ff0c2a46",
"metadata": {
"editable": true
},
@@ -546,7 +557,7 @@
},
{
"cell_type": "markdown",
"id": "0a45f635",
"id": "4c8ea78a",
"metadata": {
"editable": true
},
@@ -558,7 +569,7 @@
},
{
"cell_type": "markdown",
"id": "ac85531a",
"id": "1d119b3e",
"metadata": {
"editable": true
},
@@ -577,7 +588,7 @@
},
{
"cell_type": "markdown",
"id": "69061e11",
"id": "b9782b21",
"metadata": {
"editable": true
},
@@ -589,7 +600,7 @@
},
{
"cell_type": "markdown",
"id": "b0ed607f",
"id": "457bd0ae",
"metadata": {
"editable": true
},
@@ -603,7 +614,7 @@
},
{
"cell_type": "markdown",
"id": "3ab15499",
"id": "fbc011e0",
"metadata": {
"editable": true
},
@@ -615,7 +626,7 @@
},
{
"cell_type": "markdown",
"id": "f1c5e132",
"id": "5bb40600",
"metadata": {
"editable": true
},
@@ -625,7 +636,7 @@
},
{
"cell_type": "markdown",
"id": "60f2d431",
"id": "e5aebe0a",
"metadata": {
"editable": true
},
@@ -637,7 +648,7 @@
},
{
"cell_type": "markdown",
"id": "572f071d",
"id": "6f243211",
"metadata": {
"editable": true
},
@@ -647,7 +658,7 @@
},
{
"cell_type": "markdown",
"id": "89eb115c",
"id": "850e1403",
"metadata": {
"editable": true
},
@@ -659,7 +670,7 @@
},
{
"cell_type": "markdown",
"id": "9061bf93",
"id": "86066fab",
"metadata": {
"editable": true
},
@@ -678,32 +689,30 @@
},
{
"cell_type": "markdown",
"id": "1df7ce8d",
"id": "aedb0de8",
"metadata": {
"editable": true
},
"source": [
"### Part f): Cross-validation as resampling techniques, adding more complexity\n",
"\n",
"The aim here is to write your own code for another widely popular\n",
"The aim here is to implement another widely popular\n",
"resampling technique, the so-called cross-validation method. \n",
"\n",
"Implement the $k$-fold cross-validation algorithm (write your own\n",
"code) and evaluate again the MSE function resulting\n",
"from the test folds. You can compare your own code with that from\n",
"**Scikit-Learn** if needed. \n",
"code or use the functionality of **Scikit-Learn**) and evaluate again the MSE function resulting\n",
"from the test folds. \n",
"\n",
"Compare the MSE you get from your cross-validation code with the one\n",
"you got from your **bootstrap** code. Comment your results. Try $5-10$\n",
"folds. You can also compare your own cross-validation code with the\n",
"one provided by **Scikit-Learn**.\n",
"folds. \n",
"\n",
"In addition to using the ordinary least squares method, you should include both Ridge and Lasso regression."
]
},
{
"cell_type": "markdown",
"id": "f99f3b0c",
"id": "09e42708",
"metadata": {
"editable": true
},
@@ -731,7 +740,7 @@
{
"cell_type": "code",
"execution_count": 2,
"id": "d9d2d89a",
"id": "a7412176",
"metadata": {
"collapsed": false,
"editable": true
@@ -743,7 +752,7 @@
},
{
"cell_type": "markdown",
"id": "59f5b1a6",
"id": "2462a733",
"metadata": {
"editable": true
},
@@ -755,7 +764,7 @@
{
"cell_type": "code",
"execution_count": 3,
"id": "ab7fd531",
"id": "e299ff99",
"metadata": {
"collapsed": false,
"editable": true
@@ -781,7 +790,7 @@
},
{
"cell_type": "markdown",
"id": "271b3252",
"id": "58bfbdc9",
"metadata": {
"editable": true
},
@@ -806,7 +815,7 @@
},
{
"cell_type": "markdown",
"id": "6555e3ff",
"id": "5c69b9d7",
"metadata": {
"editable": true
},
@@ -820,7 +829,7 @@
},
{
"cell_type": "markdown",
"id": "8289021b",
"id": "a92b1a41",
"metadata": {
"editable": true
},
@@ -850,7 +859,7 @@
},
{
"cell_type": "markdown",
"id": "4349e2d3",
"id": "3da35987",
"metadata": {
"editable": true
},
@@ -872,7 +881,7 @@
},
{
"cell_type": "markdown",
"id": "274e1581",
"id": "c03bf204",
"metadata": {
"editable": true
},
doc/LectureNotes/_build/html/genindex.html
+1 -1
View File
@@ -282,7 +282,7 @@ const thebe_selector_output = ".output, .cell_output"
</p>
<ul class="nav bd-sidenav">
<li class="toctree-l1">
<a class="reference internal" href="Project1.html">
<a class="reference internal" href="project1.html">
Project 1 on Machine Learning, deadline October 7 (midnight), 2024
</a>
</li>
doc/LectureNotes/_build/html/intro.html
+1 -1
View File
@@ -283,7 +283,7 @@ const thebe_selector_output = ".output, .cell_output"
</p>
<ul class="nav bd-sidenav">
<li class="toctree-l1">
<a class="reference internal" href="Project1.html">
<a class="reference internal" href="project1.html">
Project 1 on Machine Learning, deadline October 7 (midnight), 2024
</a>
</li>
doc/LectureNotes/_build/html/objects.inv
BIN
View File
Binary file not shown.
doc/LectureNotes/_build/html/project1.html
+21 -22
View File
@@ -330,7 +330,7 @@ const thebe_selector_output = ".output, .cell_output"
<!-- ipynb file if we had a myst markdown file -->
<!-- Download raw file -->
<a class="dropdown-buttons" href="_sources/Project1.ipynb"><button type="button"
<a class="dropdown-buttons" href="_sources/project1.ipynb"><button type="button"
class="btn btn-secondary topbarbtn" title="Download source file" data-toggle="tooltip"
data-placement="left">.ipynb</button></a>
<!-- Download PDF via print -->
@@ -541,33 +541,32 @@ doconce format html Project1.do.txt -->
<p>Date: <strong>September 2</strong></p>
<div class="section" id="preamble-note-on-writing-reports-using-reference-material-ai-and-other-tools">
<h2>Preamble: Note on writing reports, using reference material, AI and other tools<a class="headerlink" href="#preamble-note-on-writing-reports-using-reference-material-ai-and-other-tools" title="Permalink to this headline"></a></h2>
<p>We want you to answer the three different projects by handing reports written like a standard scientific/technical report.
The link at <a class="reference external" href="https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ProjectWriting/projectwriting.ipynb">https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ProjectWriting/projectwriting.ipynb</a> gives some guidance. See also the grading suggestion at <a class="reference external" href="https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md">https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md</a>.</p>
<p>When using codes from different sources that you have not developed yourself,
you should refer to these in the bibliography of your report, indicating wherefrom you
<p>We want you to answer the three different projects by handing in
reports written like a standard scientific/technical report. The link
at
<a class="reference external" href="https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ProjectWriting/projectwriting.ipynb">https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ProjectWriting/projectwriting.ipynb</a>
gives some guidance. See also the grading suggestion at
<a class="reference external" href="https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md">https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md</a>.</p>
<p>Furthermore, at
<a class="reference external" href="https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/">https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/</a>
you can find examples of previous reports. How to write reports will
also be discussed during the various lab sessions. Please do ask us if you are in doubt.</p>
<p>When using codes and material from other sources, you should refer to these in the bibliography of your report, indicating wherefrom you for example
got the code, whether this is from the lecture notes, softwares like
Scikit-Learn, TensorFlow, PyTorch or other sources. These should
always be cited correctly. How to cite some of the libraries is often
indicated from their corresponding GitHub sites or websites, see for example how to cite Scikit-Learn at <a class="reference external" href="https://scikit-learn.org/dev/about.html">https://scikit-learn.org/dev/about.html</a>.</p>
<p>We enocurage you to use tools like
<a class="reference external" href="https://openai.com/chatgpt/">ChatGPT</a> in writing the report. If you use for example ChatGPT,
<a class="reference external" href="https://openai.com/chatgpt/">ChatGPT</a> or similar in writing the report. If you use for example ChatGPT,
please do cite it properly and include (if possible) your questions and answers as an addition to the report. This can
be uplodaed to for example your website, GitHub/GitLab or similar as supplemental material.</p>
<p>On scaling, we recommend reading the following section from the scikit-learn software description, see <a class="reference external" href="https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html#plot-all-scaling-standard-scaler-section">https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html#plot-all-scaling-standard-scaler-section</a></p>
be uploaded to for example your website, GitHub/GitLab or similar as supplemental material.</p>
</div>
<div class="section" id="regression-analysis-and-resampling-methods">
<h2>Regression analysis and resampling methods<a class="headerlink" href="#regression-analysis-and-resampling-methods" title="Permalink to this headline"></a></h2>
<p>The main aim of this project is to study in more detail various
regression methods, including the Ordinary Least Squares (OLS) method.
In addition to the scientific part, in this course we want also to
give you an experience in writing scientific reports. The format for
the delivery of your answers is namely that of a scientific report. At
for example
<a class="reference external" href="https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md">https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md</a>
we detail how to write a report. Furthermore, at
<a class="reference external" href="https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/">https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/</a>
you can find examples of previous reports. How to write reports will
also be discussed during the various lab sessions.</p>
give you an experience in writing scientific reports.</p>
<p><strong>A small recommendation when developing the codes here</strong>. Instead of
jumping on to the two-dimensional function described below, we
recommend to do the code development and testing with a simpler
@@ -682,6 +681,7 @@ which polynomial fits the data best.</p>
</div>
</div>
</div>
<p>If you wish to compare your results with other on the Franke function or other popular functions tested with linear regression, see the list in Figure 1 of the article by Cook et al at <a class="reference external" href="https://arxiv.org/abs/2401.11694">https://arxiv.org/abs/2401.11694</a>.</p>
</div>
<div class="section" id="part-a-ordinary-least-square-ols-on-the-franke-function">
<h3>Part a) : Ordinary Least Square (OLS) on the Franke function<a class="headerlink" href="#part-a-ordinary-least-square-ols-on-the-franke-function" title="Permalink to this headline"></a></h3>
@@ -728,6 +728,7 @@ data and say test data. An accepted rule of thumb is to use
approximately <span class="math notranslate nohighlight">\(2/3\)</span> to <span class="math notranslate nohighlight">\(4/5\)</span> of the data as training data.</p>
<p>You can easily reuse the solutions to your exercises from week 35 and week 36.
See also the lecture slides from week 35 and week 36.</p>
<p>On scaling, we recommend reading the following section from the scikit-learn software description, see <a class="reference external" href="https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html#plot-all-scaling-standard-scaler-section">https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html#plot-all-scaling-standard-scaler-section</a>.</p>
</div>
<div class="section" id="part-b-adding-ridge-regression-for-the-franke-function">
<h3>Part b): Adding Ridge regression for the Franke function<a class="headerlink" href="#part-b-adding-ridge-regression-for-the-franke-function" title="Permalink to this headline"></a></h3>
@@ -854,16 +855,14 @@ You can follow the code example in the jupyter-book at <a class="reference exter
</div>
<div class="section" id="part-f-cross-validation-as-resampling-techniques-adding-more-complexity">
<h3>Part f): Cross-validation as resampling techniques, adding more complexity<a class="headerlink" href="#part-f-cross-validation-as-resampling-techniques-adding-more-complexity" title="Permalink to this headline"></a></h3>
<p>The aim here is to write your own code for another widely popular
<p>The aim here is to implement another widely popular
resampling technique, the so-called cross-validation method.</p>
<p>Implement the <span class="math notranslate nohighlight">\(k\)</span>-fold cross-validation algorithm (write your own
code) and evaluate again the MSE function resulting
from the test folds. You can compare your own code with that from
<strong>Scikit-Learn</strong> if needed.</p>
code or use the functionality of <strong>Scikit-Learn</strong>) and evaluate again the MSE function resulting
from the test folds.</p>
<p>Compare the MSE you get from your cross-validation code with the one
you got from your <strong>bootstrap</strong> code. Comment your results. Try <span class="math notranslate nohighlight">\(5-10\)</span>
folds. You can also compare your own cross-validation code with the
one provided by <strong>Scikit-Learn</strong>.</p>
folds.</p>
<p>In addition to using the ordinary least squares method, you should include both Ridge and Lasso regression.</p>
</div>
<div class="section" id="part-g-analysis-of-real-data">
doc/LectureNotes/_build/html/search.html
+1 -1
View File
@@ -288,7 +288,7 @@ const thebe_selector_output = ".output, .cell_output"
</p>
<ul class="nav bd-sidenav">
<li class="toctree-l1">
<a class="reference internal" href="Project1.html">
<a class="reference internal" href="project1.html">
Project 1 on Machine Learning, deadline October 7 (midnight), 2024
</a>
</li>
doc/LectureNotes/_build/html/searchindex.js
+1 -1
View File
File diff suppressed because one or more lines are too long
doc/LectureNotes/_build/jupyter_execute/project1.ipynb
+83 -74
View File
@@ -2,7 +2,7 @@
"cells": [
{
"cell_type": "markdown",
"id": "f29d1e5a",
"id": "6c8c59f8",
"metadata": {
"editable": true
},
@@ -14,7 +14,7 @@
},
{
"cell_type": "markdown",
"id": "60f7a134",
"id": "24af4cf5",
"metadata": {
"editable": true
},
@@ -27,34 +27,40 @@
},
{
"cell_type": "markdown",
"id": "43b931d0",
"id": "2971d68f",
"metadata": {
"editable": true
},
"source": [
"## Preamble: Note on writing reports, using reference material, AI and other tools\n",
"\n",
"We want you to answer the three different projects by handing reports written like a standard scientific/technical report.\n",
"The link at <https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ProjectWriting/projectwriting.ipynb> gives some guidance. See also the grading suggestion at <https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md>.\n",
"We want you to answer the three different projects by handing in\n",
"reports written like a standard scientific/technical report. The link\n",
"at\n",
"<https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ProjectWriting/projectwriting.ipynb>\n",
"gives some guidance. See also the grading suggestion at\n",
"<https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md>.\n",
"\n",
"When using codes from different sources that you have not developed yourself,\n",
"you should refer to these in the bibliography of your report, indicating wherefrom you\n",
"Furthermore, at\n",
"<https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/>\n",
"you can find examples of previous reports. How to write reports will\n",
"also be discussed during the various lab sessions. Please do ask us if you are in doubt.\n",
"\n",
"When using codes and material from other sources, you should refer to these in the bibliography of your report, indicating wherefrom you for example\n",
"got the code, whether this is from the lecture notes, softwares like\n",
"Scikit-Learn, TensorFlow, PyTorch or other sources. These should\n",
"always be cited correctly. How to cite some of the libraries is often\n",
"indicated from their corresponding GitHub sites or websites, see for example how to cite Scikit-Learn at <https://scikit-learn.org/dev/about.html>. \n",
"\n",
"We enocurage you to use tools like\n",
"[ChatGPT](https://openai.com/chatgpt/) in writing the report. If you use for example ChatGPT,\n",
"[ChatGPT](https://openai.com/chatgpt/) or similar in writing the report. If you use for example ChatGPT,\n",
"please do cite it properly and include (if possible) your questions and answers as an addition to the report. This can\n",
"be uplodaed to for example your website, GitHub/GitLab or similar as supplemental material.\n",
"\n",
"On scaling, we recommend reading the following section from the scikit-learn software description, see <https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html#plot-all-scaling-standard-scaler-section>"
"be uploaded to for example your website, GitHub/GitLab or similar as supplemental material."
]
},
{
"cell_type": "markdown",
"id": "131f84dc",
"id": "eba3be6a",
"metadata": {
"editable": true
},
@@ -64,14 +70,7 @@
"The main aim of this project is to study in more detail various\n",
"regression methods, including the Ordinary Least Squares (OLS) method.\n",
"In addition to the scientific part, in this course we want also to\n",
"give you an experience in writing scientific reports. The format for\n",
"the delivery of your answers is namely that of a scientific report. At\n",
"for example\n",
"<https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md>\n",
"we detail how to write a report. Furthermore, at\n",
"<https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/>\n",
"you can find examples of previous reports. How to write reports will\n",
"also be discussed during the various lab sessions.\n",
"give you an experience in writing scientific reports.\n",
"\n",
"**A small recommendation when developing the codes here**. Instead of\n",
"jumping on to the two-dimensional function described below, we\n",
@@ -90,7 +89,7 @@
},
{
"cell_type": "markdown",
"id": "335be2ea",
"id": "50da25e5",
"metadata": {
"editable": true
},
@@ -112,7 +111,7 @@
},
{
"cell_type": "markdown",
"id": "a77e527b",
"id": "7757fb0c",
"metadata": {
"editable": true
},
@@ -127,7 +126,7 @@
},
{
"cell_type": "markdown",
"id": "071b756f",
"id": "83fbdb79",
"metadata": {
"editable": true
},
@@ -160,7 +159,7 @@
{
"cell_type": "code",
"execution_count": 1,
"id": "eae160df",
"id": "39b900ad",
"metadata": {
"collapsed": false,
"editable": true
@@ -233,7 +232,17 @@
},
{
"cell_type": "markdown",
"id": "c2cfaf54",
"id": "ef99fea4",
"metadata": {
"editable": true
},
"source": [
"If you wish to compare your results with other on the Franke function or other popular functions tested with linear regression, see the list in Figure 1 of the article by Cook et al at <https://arxiv.org/abs/2401.11694>."
]
},
{
"cell_type": "markdown",
"id": "b1a7f51d",
"metadata": {
"editable": true
},
@@ -255,7 +264,7 @@
},
{
"cell_type": "markdown",
"id": "a7cd4d8f",
"id": "d4bd58e3",
"metadata": {
"editable": true
},
@@ -268,7 +277,7 @@
},
{
"cell_type": "markdown",
"id": "c0bbdb51",
"id": "32f767b4",
"metadata": {
"editable": true
},
@@ -280,7 +289,7 @@
},
{
"cell_type": "markdown",
"id": "ddee9b4b",
"id": "bae0d9d6",
"metadata": {
"editable": true
},
@@ -292,7 +301,7 @@
},
{
"cell_type": "markdown",
"id": "6f220536",
"id": "8f0b2ec9",
"metadata": {
"editable": true
},
@@ -302,7 +311,7 @@
},
{
"cell_type": "markdown",
"id": "feb5938f",
"id": "8a455920",
"metadata": {
"editable": true
},
@@ -314,7 +323,7 @@
},
{
"cell_type": "markdown",
"id": "8ca0b371",
"id": "3f7ffe3e",
"metadata": {
"editable": true
},
@@ -338,12 +347,14 @@
"approximately $2/3$ to $4/5$ of the data as training data.\n",
"\n",
"You can easily reuse the solutions to your exercises from week 35 and week 36.\n",
"See also the lecture slides from week 35 and week 36."
"See also the lecture slides from week 35 and week 36.\n",
"\n",
"On scaling, we recommend reading the following section from the scikit-learn software description, see <https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html#plot-all-scaling-standard-scaler-section>."
]
},
{
"cell_type": "markdown",
"id": "17494425",
"id": "cf4d70a4",
"metadata": {
"editable": true
},
@@ -361,7 +372,7 @@
},
{
"cell_type": "markdown",
"id": "79411fbf",
"id": "8cc60702",
"metadata": {
"editable": true
},
@@ -378,7 +389,7 @@
},
{
"cell_type": "markdown",
"id": "0de10c86",
"id": "bd9d1dd3",
"metadata": {
"editable": true
},
@@ -394,7 +405,7 @@
},
{
"cell_type": "markdown",
"id": "b284b2d9",
"id": "03fab7b5",
"metadata": {
"editable": true
},
@@ -406,7 +417,7 @@
},
{
"cell_type": "markdown",
"id": "3a862737",
"id": "009b7fb9",
"metadata": {
"editable": true
},
@@ -417,7 +428,7 @@
},
{
"cell_type": "markdown",
"id": "8ca87494",
"id": "5bf0a0d5",
"metadata": {
"editable": true
},
@@ -429,7 +440,7 @@
},
{
"cell_type": "markdown",
"id": "a253c39b",
"id": "52c48acb",
"metadata": {
"editable": true
},
@@ -441,7 +452,7 @@
},
{
"cell_type": "markdown",
"id": "9c38fb19",
"id": "3158357a",
"metadata": {
"editable": true
},
@@ -453,7 +464,7 @@
},
{
"cell_type": "markdown",
"id": "8d59d872",
"id": "021253bc",
"metadata": {
"editable": true
},
@@ -464,7 +475,7 @@
},
{
"cell_type": "markdown",
"id": "0efd8089",
"id": "9e89d5fe",
"metadata": {
"editable": true
},
@@ -476,7 +487,7 @@
},
{
"cell_type": "markdown",
"id": "1a8121b6",
"id": "5f79916c",
"metadata": {
"editable": true
},
@@ -489,7 +500,7 @@
},
{
"cell_type": "markdown",
"id": "544da2b4",
"id": "a6e62eff",
"metadata": {
"editable": true
},
@@ -501,7 +512,7 @@
},
{
"cell_type": "markdown",
"id": "73fd411a",
"id": "7e833f14",
"metadata": {
"editable": true
},
@@ -511,7 +522,7 @@
},
{
"cell_type": "markdown",
"id": "b7160507",
"id": "14ef5a97",
"metadata": {
"editable": true
},
@@ -523,7 +534,7 @@
},
{
"cell_type": "markdown",
"id": "b18ae3c3",
"id": "a9443b1d",
"metadata": {
"editable": true
},
@@ -534,7 +545,7 @@
},
{
"cell_type": "markdown",
"id": "a57a5e8c",
"id": "ff0c2a46",
"metadata": {
"editable": true
},
@@ -567,7 +578,7 @@
},
{
"cell_type": "markdown",
"id": "0a45f635",
"id": "4c8ea78a",
"metadata": {
"editable": true
},
@@ -579,7 +590,7 @@
},
{
"cell_type": "markdown",
"id": "ac85531a",
"id": "1d119b3e",
"metadata": {
"editable": true
},
@@ -598,7 +609,7 @@
},
{
"cell_type": "markdown",
"id": "69061e11",
"id": "b9782b21",
"metadata": {
"editable": true
},
@@ -610,7 +621,7 @@
},
{
"cell_type": "markdown",
"id": "b0ed607f",
"id": "457bd0ae",
"metadata": {
"editable": true
},
@@ -624,7 +635,7 @@
},
{
"cell_type": "markdown",
"id": "3ab15499",
"id": "fbc011e0",
"metadata": {
"editable": true
},
@@ -636,7 +647,7 @@
},
{
"cell_type": "markdown",
"id": "f1c5e132",
"id": "5bb40600",
"metadata": {
"editable": true
},
@@ -646,7 +657,7 @@
},
{
"cell_type": "markdown",
"id": "60f2d431",
"id": "e5aebe0a",
"metadata": {
"editable": true
},
@@ -658,7 +669,7 @@
},
{
"cell_type": "markdown",
"id": "572f071d",
"id": "6f243211",
"metadata": {
"editable": true
},
@@ -668,7 +679,7 @@
},
{
"cell_type": "markdown",
"id": "89eb115c",
"id": "850e1403",
"metadata": {
"editable": true
},
@@ -680,7 +691,7 @@
},
{
"cell_type": "markdown",
"id": "9061bf93",
"id": "86066fab",
"metadata": {
"editable": true
},
@@ -699,32 +710,30 @@
},
{
"cell_type": "markdown",
"id": "1df7ce8d",
"id": "aedb0de8",
"metadata": {
"editable": true
},
"source": [
"### Part f): Cross-validation as resampling techniques, adding more complexity\n",
"\n",
"The aim here is to write your own code for another widely popular\n",
"The aim here is to implement another widely popular\n",
"resampling technique, the so-called cross-validation method. \n",
"\n",
"Implement the $k$-fold cross-validation algorithm (write your own\n",
"code) and evaluate again the MSE function resulting\n",
"from the test folds. You can compare your own code with that from\n",
"**Scikit-Learn** if needed. \n",
"code or use the functionality of **Scikit-Learn**) and evaluate again the MSE function resulting\n",
"from the test folds. \n",
"\n",
"Compare the MSE you get from your cross-validation code with the one\n",
"you got from your **bootstrap** code. Comment your results. Try $5-10$\n",
"folds. You can also compare your own cross-validation code with the\n",
"one provided by **Scikit-Learn**.\n",
"folds. \n",
"\n",
"In addition to using the ordinary least squares method, you should include both Ridge and Lasso regression."
]
},
{
"cell_type": "markdown",
"id": "f99f3b0c",
"id": "09e42708",
"metadata": {
"editable": true
},
@@ -752,7 +761,7 @@
{
"cell_type": "code",
"execution_count": 2,
"id": "d9d2d89a",
"id": "a7412176",
"metadata": {
"collapsed": false,
"editable": true
@@ -764,7 +773,7 @@
},
{
"cell_type": "markdown",
"id": "59f5b1a6",
"id": "2462a733",
"metadata": {
"editable": true
},
@@ -776,7 +785,7 @@
{
"cell_type": "code",
"execution_count": 3,
"id": "ab7fd531",
"id": "e299ff99",
"metadata": {
"collapsed": false,
"editable": true
@@ -802,7 +811,7 @@
},
{
"cell_type": "markdown",
"id": "271b3252",
"id": "58bfbdc9",
"metadata": {
"editable": true
},
@@ -827,7 +836,7 @@
},
{
"cell_type": "markdown",
"id": "6555e3ff",
"id": "5c69b9d7",
"metadata": {
"editable": true
},
@@ -841,7 +850,7 @@
},
{
"cell_type": "markdown",
"id": "8289021b",
"id": "a92b1a41",
"metadata": {
"editable": true
},
@@ -871,7 +880,7 @@
},
{
"cell_type": "markdown",
"id": "4349e2d3",
"id": "3da35987",
"metadata": {
"editable": true
},
@@ -893,7 +902,7 @@
},
{
"cell_type": "markdown",
"id": "274e1581",
"id": "c03bf204",
"metadata": {
"editable": true
},
doc/LectureNotes/_build/jupyter_execute/project1.py
+23 -22
View File
@@ -12,36 +12,35 @@
# ## Preamble: Note on writing reports, using reference material, AI and other tools
#
# We want you to answer the three different projects by handing reports written like a standard scientific/technical report.
# The link at <https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ProjectWriting/projectwriting.ipynb> gives some guidance. See also the grading suggestion at <https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md>.
# We want you to answer the three different projects by handing in
# reports written like a standard scientific/technical report. The link
# at
# <https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ProjectWriting/projectwriting.ipynb>
# gives some guidance. See also the grading suggestion at
# <https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md>.
#
# When using codes from different sources that you have not developed yourself,
# you should refer to these in the bibliography of your report, indicating wherefrom you
# Furthermore, at
# <https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/>
# you can find examples of previous reports. How to write reports will
# also be discussed during the various lab sessions. Please do ask us if you are in doubt.
#
# When using codes and material from other sources, you should refer to these in the bibliography of your report, indicating wherefrom you for example
# got the code, whether this is from the lecture notes, softwares like
# Scikit-Learn, TensorFlow, PyTorch or other sources. These should
# always be cited correctly. How to cite some of the libraries is often
# indicated from their corresponding GitHub sites or websites, see for example how to cite Scikit-Learn at <https://scikit-learn.org/dev/about.html>.
#
# We enocurage you to use tools like
# [ChatGPT](https://openai.com/chatgpt/) in writing the report. If you use for example ChatGPT,
# [ChatGPT](https://openai.com/chatgpt/) or similar in writing the report. If you use for example ChatGPT,
# please do cite it properly and include (if possible) your questions and answers as an addition to the report. This can
# be uplodaed to for example your website, GitHub/GitLab or similar as supplemental material.
#
# On scaling, we recommend reading the following section from the scikit-learn software description, see <https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html#plot-all-scaling-standard-scaler-section>
# be uploaded to for example your website, GitHub/GitLab or similar as supplemental material.
# ## Regression analysis and resampling methods
#
# The main aim of this project is to study in more detail various
# regression methods, including the Ordinary Least Squares (OLS) method.
# In addition to the scientific part, in this course we want also to
# give you an experience in writing scientific reports. The format for
# the delivery of your answers is namely that of a scientific report. At
# for example
# <https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md>
# we detail how to write a report. Furthermore, at
# <https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/>
# you can find examples of previous reports. How to write reports will
# also be discussed during the various lab sessions.
# give you an experience in writing scientific reports.
#
# **A small recommendation when developing the codes here**. Instead of
# jumping on to the two-dimensional function described below, we
@@ -148,6 +147,8 @@ fig.colorbar(surf, shrink=0.5, aspect=5)
plt.show()
# If you wish to compare your results with other on the Franke function or other popular functions tested with linear regression, see the list in Figure 1 of the article by Cook et al at <https://arxiv.org/abs/2401.11694>.
# ### Part a) : Ordinary Least Square (OLS) on the Franke function
#
# We will generate our own dataset for a function
@@ -201,6 +202,8 @@ plt.show()
#
# You can easily reuse the solutions to your exercises from week 35 and week 36.
# See also the lecture slides from week 35 and week 36.
#
# On scaling, we recommend reading the following section from the scikit-learn software description, see <https://scikit-learn.org/stable/auto_examples/preprocessing/plot_all_scaling.html#plot-all-scaling-standard-scaler-section>.
# ### Part b): Adding Ridge regression for the Franke function
#
@@ -352,18 +355,16 @@ plt.show()
# ### Part f): Cross-validation as resampling techniques, adding more complexity
#
# The aim here is to write your own code for another widely popular
# The aim here is to implement another widely popular
# resampling technique, the so-called cross-validation method.
#
# Implement the $k$-fold cross-validation algorithm (write your own
# code) and evaluate again the MSE function resulting
# from the test folds. You can compare your own code with that from
# **Scikit-Learn** if needed.
# code or use the functionality of **Scikit-Learn**) and evaluate again the MSE function resulting
# from the test folds.
#
# Compare the MSE you get from your cross-validation code with the one
# you got from your **bootstrap** code. Comment your results. Try $5-10$
# folds. You can also compare your own cross-validation code with the
# one provided by **Scikit-Learn**.
# folds.
#
# In addition to using the ordinary least squares method, you should include both Ridge and Lasso regression.
doc/LectureNotes/_toc.yml
+1 -1
View File
@@ -50,6 +50,6 @@ parts:
- caption: Projects
numbered: false
chapters:
- file: Project1.ipynb
- file: project1.ipynb
# - file: project2.ipynb
# - file: Project3.ipynb