-which reads as finding the likelihood of an event \( y_i \) given the input variables \( \boldsymbol{X} \) and the parameters (to be determined) \( \boldsymbol{beta} \).
+which reads as finding the likelihood of an event \( y_i \) given the input variables \( \boldsymbol{X} \) and the parameters (to be determined) \( \boldsymbol{\beta} \).
Since these events are assumed to be independent and identicall distributed we can build the probability distribution function (PDF) for all possible event \( \boldsymbol{y} \) as the product of the single events, that is we have
@@ -1146,10 +1146,44 @@ is equivalent to the maximization/minimization of the function itself.
-More text will be added here.
+We could now define a new cost function to minimize, namely the negative logarithm of the above PDF
+
+
+Taking the derivative of the new cost function with respect to the parameters \( \beta \) we recognize our familiar OLS equation, namely
+
+ Friday September 10
+A new Cost Function
+$$
+C(\boldsymbol{\beta}=-\log{\prod_{i=0}^{n-1}p(y_i\vert \boldsymbol{X};\boldsymbol{\beta})}=-\sum_{i=0}^{n-1}\log{p(y_i\vert \boldsymbol{X};\boldsymbol{\beta})},
+$$
+
+
+which becomes
+
+$$
+C(\boldsymbol{\beta}=\frac{n}{2}\log{2\pi\sigma^2}+\frac{\vert\vert (\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})\vert\vert_2^2}{2\sigma^2}.
+$$
+
+
+
+$$
+\boldsymbol{X}^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right) =0,
+$$
+
+
+which leads to
+
+$$
+\hat{\boldsymbol{\beta}}_{mathrm{OLS}}=\left(\boldsymbol{X}^T\boldsymbol{X}\right^{-1}\boldsymbol{X}^T\boldsymbol{y}!
+$$
+
+Friday September 10
Since these events are assumed to be independent and identicall distributed we can build the probability distribution function (PDF) for all possible event \( \boldsymbol{y} \) as the product of the single events, that is we have @@ -1126,10 +1127,36 @@ is equivalent to the maximization/minimization of the function itself.
-
-More text will be added here. +We could now define a new cost function to minimize, namely the negative logarithm of the above PDF + +$$ +C(\boldsymbol{\beta}=-\log{\prod_{i=0}^{n-1}p(y_i\vert \boldsymbol{X};\boldsymbol{\beta})}=-\sum_{i=0}^{n-1}\log{p(y_i\vert \boldsymbol{X};\boldsymbol{\beta})}, +$$ + +which becomes +$$ +C(\boldsymbol{\beta}=\frac{n}{2}\log{2\pi\sigma^2}+\frac{\vert\vert (\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})\vert\vert_2^2}{2\sigma^2}. +$$ + +
+Taking the derivative of the new cost function with respect to the parameters \( \beta \) we recognize our familiar OLS equation, namely + +$$ +\boldsymbol{X}^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right) =0, +$$ + +which leads to +$$ +\hat{\boldsymbol{\beta}}_{mathrm{OLS}}=\left(\boldsymbol{X}^T\boldsymbol{X}\right^{-1}\boldsymbol{X}^T\boldsymbol{y}! +$$ + +
+
+
+
diff --git a/doc/pub/week36/html/week36.html b/doc/pub/week36/html/week36.html
index c854ba4d4..828d35f4c 100644
--- a/doc/pub/week36/html/week36.html
+++ b/doc/pub/week36/html/week36.html
@@ -149,6 +149,7 @@ div { text-align: justify; text-justify: inter-word; }
2,
None,
'maximum-likelihood-estimation-mle'),
+ ('A new Cost Function', 2, None, 'a-new-cost-function'),
('Friday September 10', 2, None, 'friday-september-10'),
('Why resampling methods', 2, None, 'why-resampling-methods'),
('Resampling methods', 2, None, 'resampling-methods'),
@@ -1089,7 +1090,7 @@ $$
p(y_i\vert \boldsymbol{X};\boldsymbol{\beta})=\frac{1}{\sqrt{2\pi\sigma^2}}\exp{\left[-\frac{(y_i-\boldsymbol{X}_{i,*}\boldsymbol{\beta})^2}{2\sigma^2}\right]},
$$
-which reads as finding the likelihood of an event \( y_i \) given the input variables \( \boldsymbol{X} \) and the parameters (to be determined) \( \boldsymbol{beta} \).
+which reads as finding the likelihood of an event \( y_i \) given the input variables \( \boldsymbol{X} \) and the parameters (to be determined) \( \boldsymbol{\beta} \).
Since these events are assumed to be independent and identicall distributed we can build the probability distribution function (PDF) for all possible event \( \boldsymbol{y} \) as the product of the single events, that is we have @@ -1131,10 +1132,36 @@ is equivalent to the maximization/minimization of the function itself.
-
-More text will be added here. +We could now define a new cost function to minimize, namely the negative logarithm of the above PDF + +$$ +C(\boldsymbol{\beta}=-\log{\prod_{i=0}^{n-1}p(y_i\vert \boldsymbol{X};\boldsymbol{\beta})}=-\sum_{i=0}^{n-1}\log{p(y_i\vert \boldsymbol{X};\boldsymbol{\beta})}, +$$ + +which becomes +$$ +C(\boldsymbol{\beta}=\frac{n}{2}\log{2\pi\sigma^2}+\frac{\vert\vert (\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta})\vert\vert_2^2}{2\sigma^2}. +$$ + +
+Taking the derivative of the new cost function with respect to the parameters \( \beta \) we recognize our familiar OLS equation, namely + +$$ +\boldsymbol{X}^T\left(\boldsymbol{y}-\boldsymbol{X}\boldsymbol{\beta}\right) =0, +$$ + +which leads to +$$ +\hat{\boldsymbol{\beta}}_{mathrm{OLS}}=\left(\boldsymbol{X}^T\boldsymbol{X}\right^{-1}\boldsymbol{X}^T\boldsymbol{y}! +$$ + +
+
+
+
diff --git a/doc/pub/week36/ipynb/ipynb-week36-src.tar.gz b/doc/pub/week36/ipynb/ipynb-week36-src.tar.gz
index f055db04d..e45110d14 100644
Binary files a/doc/pub/week36/ipynb/ipynb-week36-src.tar.gz and b/doc/pub/week36/ipynb/ipynb-week36-src.tar.gz differ
diff --git a/doc/pub/week36/ipynb/week36.ipynb b/doc/pub/week36/ipynb/week36.ipynb
index f16f43789..919e6242a 100644
--- a/doc/pub/week36/ipynb/week36.ipynb
+++ b/doc/pub/week36/ipynb/week36.ipynb
@@ -1408,7 +1408,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "which reads as finding the likelihood of an event $y_i$ given the input variables $\\boldsymbol{X}$ and the parameters (to be determined) $\\boldsymbol{beta}$.\n",
+ "which reads as finding the likelihood of an event $y_i$ given the input variables $\\boldsymbol{X}$ and the parameters (to be determined) $\\boldsymbol{\\beta}$.\n",
"\n",
"Since these events are assumed to be independent and identicall distributed we can build the probability distribution function (PDF) for all possible event $\\boldsymbol{y}$ as the product of the single events, that is we have"
]
@@ -1453,10 +1453,75 @@
"\n",
"\n",
"\n",
+ "## A new Cost Function\n",
"\n",
+ "We could now define a new cost function to minimize, namely the negative logarithm of the above PDF"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "$$\n",
+ "C(\\boldsymbol{\\beta}=-\\log{\\prod_{i=0}^{n-1}p(y_i\\vert \\boldsymbol{X};\\boldsymbol{\\beta})}=-\\sum_{i=0}^{n-1}\\log{p(y_i\\vert \\boldsymbol{X};\\boldsymbol{\\beta})},\n",
+ "$$"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "which becomes"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "$$\n",
+ "C(\\boldsymbol{\\beta}=\\frac{n}{2}\\log{2\\pi\\sigma^2}+\\frac{\\vert\\vert (\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta})\\vert\\vert_2^2}{2\\sigma^2}.\n",
+ "$$"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Taking the derivative of the *new* cost function with respect to the parameters $\\beta$ we recognize our familiar OLS equation, namely"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "$$\n",
+ "\\boldsymbol{X}^T\\left(\\boldsymbol{y}-\\boldsymbol{X}\\boldsymbol{\\beta}\\right) =0,\n",
+ "$$"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "which leads to"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "$$\n",
+ "\\hat{\\boldsymbol{\\beta}}_{mathrm{OLS}}=\\left(\\boldsymbol{X}^T\\boldsymbol{X}\\right^{-1}\\boldsymbol{X}^T\\boldsymbol{y}!\n",
+ "$$"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
"## Friday September 10\n",
"\n",
- "More text will be added here.\n",
+ "\n",
"\n",
"## Why resampling methods\n",
"\n",
diff --git a/doc/src/week36/week36.do.txt b/doc/src/week36/week36.do.txt
index fd5b87109..77f964392 100644
--- a/doc/src/week36/week36.do.txt
+++ b/doc/src/week36/week36.do.txt
@@ -791,7 +791,7 @@ We define this distribution as
p(y_i\vert \bm{X};\bm{\beta})=\frac{1}{\sqrt{2\pi\sigma^2}}\exp{\left[-\frac{(y_i-\bm{X}_{i,*}\bm{\beta})^2}{2\sigma^2}\right]},
\]
!et
-which reads as finding the likelihood of an event $y_i$ given the input variables $\bm{X}$ and the parameters (to be determined) $\bm{beta}$.
+which reads as finding the likelihood of an event $y_i$ given the input variables $\bm{X}$ and the parameters (to be determined) $\bm{\beta}$.
Since these events are assumed to be independent and identicall distributed we can build the probability distribution function (PDF) for all possible event $\bm{y}$ as the product of the single events, that is we have
@@ -829,11 +829,42 @@ is equivalent to the maximization/minimization of the function itself.
+!split
+===== A new Cost Function =====
+
+We could now define a new cost function to minimize, namely the negative logarithm of the above PDF
+
+!bt
+\[
+C(\bm{\beta}=-\log{\prod_{i=0}^{n-1}p(y_i\vert \bm{X};\bm{\beta})}=-\sum_{i=0}^{n-1}\log{p(y_i\vert \bm{X};\bm{\beta})},
+\]
+!et
+which becomes
+!bt
+\[
+C(\bm{\beta}=\frac{n}{2}\log{2\pi\sigma^2}+\frac{\vert\vert (\bm{y}-\bm{X}\bm{\beta})\vert\vert_2^2}{2\sigma^2}.
+\]
+!et
+
+Taking the derivative of the *new* cost function with respect to the parameters $\beta$ we recognize our familiar OLS equation, namely
+
+!bt
+\[
+\bm{X}^T\left(\bm{y}-\bm{X}\bm{\beta}\right) =0,
+\]
+!et
+which leads to
+!bt
+\[
+\hat{\bm{\beta}}_{mathrm{OLS}}=\left(\bm{X}^T\bm{X}\right^{-1}\bm{X}^T\bm{y}!
+\]
+!et
+
!split
===== Friday September 10 =====
-More text will be added here.
+
!split
===== Why resampling methods =====