lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

updates

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2025-08-18 10:47:43 +02:00
parent d4db3104e6
commit 35c45696d6
37 changed files with 5815 additions and 5344 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/LectureNotes/E2.ipynb
+46 -24
View File
@@ -146,9 +146,9 @@
"id": "f3f771de",
"metadata": {},
"source": [
"The ordinary least squares method finds the parameters $\\boldsymbol{\\beta}$ which minimizes the squared error between our model $\\boldsymbol{X\\beta}$ and the true values $\\boldsymbol{y}$.\n",
"The ordinary least squares method finds the parameters $\\boldsymbol{\\theta}$ which minimizes the squared error between our model $\\boldsymbol{X\\theta}$ and the true values $\\boldsymbol{y}$.\n",
"\n",
"To find the parameters $\\boldsymbol{\\beta}$ which minimizes this error, we take the derivative of the squared error expression with respect to $\\boldsymbol{\\beta}$, and set it equal to 0."
"To find the parameters $\\boldsymbol{\\theta}$ which minimizes this error, we take the derivative of the squared error expression with respect to $\\boldsymbol{\\theta}$, and set it equal to 0."
]
},
{
@@ -156,7 +156,7 @@
"id": "49690237",
"metadata": {},
"source": [
"**a)** Very briefly explain why the approach above finds the parameters $\\boldsymbol{\\beta}$ which minimizes this error."
"**a)** Very briefly explain why the approach above finds the parameters $\\boldsymbol{\\theta}$ which minimizes this error."
]
},
{
@@ -167,13 +167,13 @@
"We typically write the squared error as\n",
"\n",
"$$\n",
"\\vert\\vert\\boldsymbol{y} - \\boldsymbol{X\\beta}\\vert\\vert^2\n",
"\\vert\\vert\\boldsymbol{y} - \\boldsymbol{X\\theta}\\vert\\vert^2\n",
"$$\n",
"\n",
"which we can rewrite in matrix-vector form as\n",
"\n",
"$$\n",
"\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right)\n",
"\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\right)^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\theta}\\right)\n",
"$$"
]
},
@@ -182,7 +182,7 @@
"id": "8fbecf74",
"metadata": {},
"source": [
"**b)** If $\\boldsymbol{X}$ is invertible, what is the expression for the optimal parameters $\\boldsymbol{\\beta}$? (**Hint:** Don't compute any derivatives, but solve $\\boldsymbol{X\\beta}=\\boldsymbol{y}$ for $\\boldsymbol{\\beta}$)"
"**b)** If $\\boldsymbol{X}$ is invertible, what is the expression for the optimal parameters $\\boldsymbol{\\theta}$? (**Hint:** Don't compute any derivatives, but solve $\\boldsymbol{X\\theta}=\\boldsymbol{y}$ for $\\boldsymbol{\\theta}$)"
]
},
{
@@ -202,10 +202,10 @@
"id": "869fca4d",
"metadata": {},
"source": [
"**d)** Using the expression from **c)**, but substituting back in $\\boldsymbol{\\beta}$, $\\boldsymbol{y}$ and $\\boldsymbol{X}$, find the expression for the optimal parameters $\\boldsymbol{\\beta}$ in the case that $\\boldsymbol{X}$ is not invertible, but $\\boldsymbol{X^T X}$ is, which is most often the case.\n",
"**d)** Using the expression from **c)**, but substituting back in $\\boldsymbol{\\theta}$, $\\boldsymbol{y}$ and $\\boldsymbol{X}$, find the expression for the optimal parameters $\\boldsymbol{\\theta}$ in the case that $\\boldsymbol{X}$ is not invertible, but $\\boldsymbol{X^T X}$ is, which is most often the case.\n",
"\n",
"$$\n",
"\\boldsymbol{\\hat{\\beta}_{OLS}} = ...\n",
"\\boldsymbol{\\hat{\\theta}_{OLS}} = ...\n",
"$$"
]
},
@@ -222,18 +222,18 @@
"id": "5dc179f7",
"metadata": {},
"source": [
"With the expression for $\\boldsymbol{\\hat{\\beta}_{OLS}}$, you now have what you need to implement OLS regression with your input data and target data $\\boldsymbol{y}$. But before you can do that, you need to set up you input data as a feature matrix $\\boldsymbol{X}$.\n",
"With the expression for $\\boldsymbol{\\hat{\\theta}_{OLS}}$, you now have what you need to implement OLS regression with your input data and target data $\\boldsymbol{y}$. But before you can do that, you need to set up you input data as a feature matrix $\\boldsymbol{X}$.\n",
"\n",
"In a feature matrix, each row is a datapoint and each column is a feature of that data. If you want to predict someones spending based on their income and number of children, for instance, you would create a row for each person in your dataset, with the montly income and the number of children as columns.\n",
"\n",
"We typically also include an intercept in our models. The intercept is a value that is added to our prediction regardless of the value of the other features. The intercept tries to account for constant effects in our data that are not dependant on anything else. In our current example, the intercept could account for living expenses which are typical regardless of income or childcare expenses.\n",
"\n",
"We calculate the optimal intercept by including a feature with the constant value of 1 in our model, which is then multplied by some parameter $\\beta_0$ from the OLS method into the optimal intercept value (which will be $\\beta_0$). In practice, we include the intercept in our model by adding a column of ones to the start of our feature matrix."
"We calculate the optimal intercept by including a feature with the constant value of 1 in our model, which is then multplied by some parameter $\\theta_0$ from the OLS method into the optimal intercept value (which will be $\\theta_0$). In practice, we include the intercept in our model by adding a column of ones to the start of our feature matrix."
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 1,
"id": "e5ff2a69",
"metadata": {},
"outputs": [],
@@ -243,7 +243,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 2,
"id": "a3cf2792",
"metadata": {},
"outputs": [],
@@ -264,7 +264,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 3,
"id": "5ad87a65",
"metadata": {},
"outputs": [],
@@ -285,7 +285,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 4,
"id": "8f3f68aa",
"metadata": {},
"outputs": [],
@@ -316,7 +316,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 5,
"id": "d7476c84",
"metadata": {},
"outputs": [],
@@ -336,7 +336,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 6,
"id": "91496e40",
"metadata": {},
"outputs": [],
@@ -362,7 +362,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 7,
"id": "034f502c",
"metadata": {},
"outputs": [],
@@ -380,7 +380,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 8,
"id": "29171358",
"metadata": {},
"outputs": [],
@@ -400,10 +400,21 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 9,
"id": "1e346f4c",
"metadata": {},
"outputs": [],
"outputs": [
{
"data": {
"text/plain": [
"Ellipsis"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"..."
]
@@ -418,10 +429,21 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 10,
"id": "ceb57457",
"metadata": {},
"outputs": [],
"outputs": [
{
"data": {
"text/plain": [
"Ellipsis"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"..."
]
@@ -495,7 +517,7 @@
],
"metadata": {
"kernelspec": {
"display_name": ".venv",
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
@@ -509,7 +531,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.0"
"version": "3.9.15"
}
},
"nbformat": 4,