+Our basic assumption when we derived the OLS equations was to assume +that our output is determing by a given continuous function +\( f(\boldsymbol{x}) \) and a random noise \( \boldsymbol{\epsilon} \) given by the normal +distribution with zero mean value and an undetermined variance +\( \sigma^2 \). + +
+We found above that the outputs \( \boldsymbol{y} \) have a mean value given by +\( \boldsymbol{X}\hat{\boldsymbol{\beta}} \) and variance \( \sigma^2 \). Since the entries to +the design matrix are not stocastic variables, we can assume that the +probability distribution of our targets is also a normal distribution +but now with mean value \( \boldsymbol{X}\hat{\boldsymbol{\beta}} \). This means that a +single output \( y_i \) is given by the Gaussian distribution +
+
+Our basic assumption when we derived the OLS equations was to assume +that our output is determing by a given continuous function +\( f(\boldsymbol{x}) \) and a random noise \( \boldsymbol{\epsilon} \) given by the normal +distribution with zero mean value and an undetermined variance +\( \sigma^2 \). + +
+We found above that the outputs \( \boldsymbol{y} \) have a mean value given by +\( \boldsymbol{X}\hat{\boldsymbol{\beta}} \) and variance \( \sigma^2 \). Since the entries to +the design matrix are not stocastic variables, we can assume that the +probability distribution of our targets is also a normal distribution +but now with mean value \( \boldsymbol{X}\hat{\boldsymbol{\beta}} \). This means that a +single output \( y_i \) is given by the Gaussian distribution + +
+
+
diff --git a/doc/pub/week36/html/week36.html b/doc/pub/week36/html/week36.html index db69c4b73..58b376813 100644 --- a/doc/pub/week36/html/week36.html +++ b/doc/pub/week36/html/week36.html @@ -127,6 +127,10 @@ div { text-align: justify; text-justify: inter-word; } 2, None, 'expectation-value-and-variance-for-boldsymbol-beta'), + ('Deriving OLS from a probability distribution', + 2, + None, + 'deriving-ols-from-a-probability-distribution'), ('Friday September 10', 2, None, 'friday-september-10'), ('Why resampling methods', 2, None, 'why-resampling-methods'), ('Resampling methods', 2, None, 'resampling-methods'), @@ -932,6 +936,26 @@ This means the variance we obtain with the standard OLS will always for \( \lamb
+
+Our basic assumption when we derived the OLS equations was to assume +that our output is determing by a given continuous function +\( f(\boldsymbol{x}) \) and a random noise \( \boldsymbol{\epsilon} \) given by the normal +distribution with zero mean value and an undetermined variance +\( \sigma^2 \). + +
+We found above that the outputs \( \boldsymbol{y} \) have a mean value given by +\( \boldsymbol{X}\hat{\boldsymbol{\beta}} \) and variance \( \sigma^2 \). Since the entries to +the design matrix are not stocastic variables, we can assume that the +probability distribution of our targets is also a normal distribution +but now with mean value \( \boldsymbol{X}\hat{\boldsymbol{\beta}} \). This means that a +single output \( y_i \) is given by the Gaussian distribution + +
+
+
diff --git a/doc/pub/week36/ipynb/ipynb-week36-src.tar.gz b/doc/pub/week36/ipynb/ipynb-week36-src.tar.gz index e9efb4e29..0d8bee72a 100644 Binary files a/doc/pub/week36/ipynb/ipynb-week36-src.tar.gz and b/doc/pub/week36/ipynb/ipynb-week36-src.tar.gz differ diff --git a/doc/pub/week36/ipynb/week36.ipynb b/doc/pub/week36/ipynb/week36.ipynb index 8be9302f1..70e4885f7 100644 --- a/doc/pub/week36/ipynb/week36.ipynb +++ b/doc/pub/week36/ipynb/week36.ipynb @@ -1157,6 +1157,21 @@ "This means the variance we obtain with the standard OLS will always for $\\lambda > 0$ be larger than the variance of $\\boldsymbol{\\beta}$ obtained with the Ridge estimator. This has interesting consequences when we discuss the so-called bias-variance trade-off below. \n", "\n", "\n", + "## Deriving OLS from a probability distribution\n", + "\n", + "Our basic assumption when we derived the OLS equations was to assume\n", + "that our output is determing by a given continuous function\n", + "$f(\\boldsymbol{x})$ and a random noise $\\boldsymbol{\\epsilon}$ given by the normal\n", + "distribution with zero mean value and an undetermined variance\n", + "$\\sigma^2$.\n", + "\n", + "We found above that the outputs $\\boldsymbol{y}$ have a mean value given by\n", + "$\\boldsymbol{X}\\hat{\\boldsymbol{\\beta}}$ and variance $\\sigma^2$. Since the entries to\n", + "the design matrix are not stocastic variables, we can assume that the\n", + "probability distribution of our targets is also a normal distribution\n", + "but now with mean value $\\boldsymbol{X}\\hat{\\boldsymbol{\\beta}}$. This means that a\n", + "single output $y_i$ is given by the Gaussian distribution\n", + "\n", "\n", "\n", "\n", diff --git a/doc/src/week36/week36.do.txt b/doc/src/week36/week36.do.txt index 4a47a5747..44994a66a 100644 --- a/doc/src/week36/week36.do.txt +++ b/doc/src/week36/week36.do.txt @@ -658,6 +658,22 @@ matrix product is non-negative definite. This means the variance we obtain with the standard OLS will always for $\lambda > 0$ be larger than the variance of $\bm{\beta}$ obtained with the Ridge estimator. This has interesting consequences when we discuss the so-called bias-variance trade-off below. +!split +===== Deriving OLS from a probability distribution ===== + +Our basic assumption when we derived the OLS equations was to assume +that our output is determing by a given continuous function +$f(\bm{x})$ and a random noise $\bm{\epsilon}$ given by the normal +distribution with zero mean value and an undetermined variance +$\sigma^2$. + +We found above that the outputs $\bm{y}$ have a mean value given by +$\bm{X}\hat{\bm{\beta}}$ and variance $\sigma^2$. Since the entries to +the design matrix are not stocastic variables, we can assume that the +probability distribution of our targets is also a normal distribution +but now with mean value $\bm{X}\hat{\bm{\beta}}$. This means that a +single output $y_i$ is given by the Gaussian distribution +