update of notes
This commit is contained in:
Morten Hjorth-Jensen
@@ -2,7 +2,7 @@
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "78bc86fe",
|
||||
"id": "b2cf4bdf",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -14,7 +14,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "bfac1a23",
|
||||
"id": "43316fd6",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -27,7 +27,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "248903ce",
|
||||
"id": "1f8db1a6",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -45,7 +45,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "d08c4671",
|
||||
"id": "41067a3c",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -57,7 +57,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "56f9ca3e",
|
||||
"id": "68997f6b",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -76,7 +76,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "ae36b494",
|
||||
"id": "f7e45184",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -88,7 +88,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "be3eadf1",
|
||||
"id": "4c15509e",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -97,24 +97,25 @@
|
||||
"\n",
|
||||
"Show that you can rewrite this in terms of a term which contains the variance of the model itself (the so-called variance term), a\n",
|
||||
"term which measures the deviation from the true data and the mean value of the model (the bias term) and finally the variance of the noise.\n",
|
||||
"\n",
|
||||
"That is, show that"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "ae9ebea0",
|
||||
"id": "046ec084",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=(\\mathrm{Bias}[\\tilde{y}])^2+\\mathrm{var}[\\tilde{f}]+\\sigma^2,\n",
|
||||
"\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=\\mathrm{Bias}[y]+\\mathrm{var}[\\tilde{y}]+\\sigma^2,\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "d1cbae1b",
|
||||
"id": "b2cc33db",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -124,19 +125,19 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "d2e1f899",
|
||||
"id": "7ab1bc0e",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"(\\mathrm{Bias}[\\tilde{y}])^2=\\left(\\boldsymbol{y}-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right]\\right)^2,\n",
|
||||
"\\mathrm{Bias}[y]=\\mathbb{E}\\left[\\left(\\boldsymbol{y}-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right]\\right)^2\\right],\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "0486221c",
|
||||
"id": "a8da71db",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -146,23 +147,26 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "86746df2",
|
||||
"id": "b74aa5e4",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\mathrm{var}[\\tilde{f}]=\\frac{1}{n}\\sum_i(\\tilde{y}_i-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2.\n",
|
||||
"\\mathrm{var}[\\tilde{y}]=\\frac{1}{n}\\sum_i(\\tilde{y}_i-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2.\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "9aa6d3dc",
|
||||
"id": "de559d80",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"The answer to this exercise should be included in the theory part of the report. This exercise is also part of the weekly exercises of week 37.\n",
|
||||
"Explain what the terms mean and discuss their interpretations.\n",
|
||||
"\n",
|
||||
"Explain what the terms mean and discuss their interpretations.\n",
|
||||
"\n",
|
||||
"Perform then a bias-variance analysis of a simple one-dimensional (or other models of your choice) function by\n",
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "6060626f",
|
||||
"id": "7424fea1",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -14,7 +14,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "fd9f7098",
|
||||
"id": "c9ca1d90",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -27,7 +27,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "75f7ac9b",
|
||||
"id": "3e4164b6",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -63,7 +63,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "988b2436",
|
||||
"id": "dd4dcdf5",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -85,7 +85,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "0725f6b3",
|
||||
"id": "71ddc478",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -100,7 +100,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "de583594",
|
||||
"id": "d4184afb",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -133,7 +133,7 @@
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"id": "130a5d79",
|
||||
"id": "62880fe6",
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"editable": true
|
||||
@@ -185,7 +185,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "9bf030f4",
|
||||
"id": "6f7e7c97",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -207,7 +207,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "3be47d8d",
|
||||
"id": "7377112f",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -220,7 +220,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "f7f14618",
|
||||
"id": "72f83fe7",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -232,7 +232,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "913d72b1",
|
||||
"id": "7a3578ff",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -244,7 +244,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "7a403d11",
|
||||
"id": "e0aef819",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -254,7 +254,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "108b7a27",
|
||||
"id": "c7f751af",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -266,7 +266,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "f1e64231",
|
||||
"id": "c026a4bd",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -295,7 +295,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "30bca1e7",
|
||||
"id": "2635bcbd",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -313,7 +313,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "ae6a3eeb",
|
||||
"id": "92892df2",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -330,7 +330,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "5ba7dff9",
|
||||
"id": "35987aab",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -346,7 +346,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "84ca72c8",
|
||||
"id": "e61048c2",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -358,7 +358,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "0d734865",
|
||||
"id": "5ef456d4",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -369,7 +369,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "1dac6c19",
|
||||
"id": "52e6fbad",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -381,7 +381,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "e04687a9",
|
||||
"id": "db6028a0",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -393,7 +393,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "f99dbe69",
|
||||
"id": "9477ad5e",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -405,7 +405,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "0b3e0564",
|
||||
"id": "8c2aeccd",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -416,7 +416,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "f660901d",
|
||||
"id": "6a774b74",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -428,7 +428,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "7a1c7f34",
|
||||
"id": "98fde9b7",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -441,7 +441,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "becad040",
|
||||
"id": "87a4d11d",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -453,7 +453,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "242ede2f",
|
||||
"id": "b4844f1e",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -463,7 +463,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "d4999c5f",
|
||||
"id": "f7662ade",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -475,7 +475,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "46386eef",
|
||||
"id": "1d61362e",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -486,7 +486,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "9f27e396",
|
||||
"id": "74bbb4ab",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -519,7 +519,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "902b759d",
|
||||
"id": "ac939724",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -531,7 +531,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "41a0b8d7",
|
||||
"id": "4066d012",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -550,7 +550,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "89cced85",
|
||||
"id": "7e5c5ef4",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -562,7 +562,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "22bb8b90",
|
||||
"id": "690337b9",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -576,19 +576,19 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "e88ba22d",
|
||||
"id": "92dabf3c",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=(\\mathrm{Bias}[\\tilde{y}])^2+\\mathrm{var}[\\tilde{f}]+\\sigma^2,\n",
|
||||
"\\mathbb{E}\\left[(\\boldsymbol{y}-\\boldsymbol{\\tilde{y}})^2\\right]=\\mathrm{Bias}[y]+\\mathrm{var}[\\tilde{y}]+\\sigma^2,\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "e2156ab0",
|
||||
"id": "7da90966",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -598,19 +598,19 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "7d08fd36",
|
||||
"id": "33a81d54",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"(\\mathrm{Bias}[\\tilde{y}])^2=\\left(\\boldsymbol{y}-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right]\\right)^2,\n",
|
||||
"\\mathrm{Bias}[y]=\\mathbb{E}\\left[\\left(\\boldsymbol{y}-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right]\\right)^2\\right],\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "5cdcf0c6",
|
||||
"id": "abb9bd28",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -620,19 +620,19 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "3235550b",
|
||||
"id": "5b4668fd",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\mathrm{var}[\\tilde{f}]=\\frac{1}{n}\\sum_i(\\tilde{y}_i-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2.\n",
|
||||
"\\mathrm{var}[\\tilde{y}]=\\frac{1}{n}\\sum_i(\\tilde{y}_i-\\mathbb{E}\\left[\\boldsymbol{\\tilde{y}}\\right])^2.\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "813e8659",
|
||||
"id": "b453f8fc",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -651,7 +651,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "adf1396f",
|
||||
"id": "6fe323b0",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -676,7 +676,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "a7cb0fd1",
|
||||
"id": "5fd2f486",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -704,7 +704,7 @@
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"id": "3be6c174",
|
||||
"id": "e88e39ea",
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"editable": true
|
||||
@@ -716,7 +716,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "9544f6e7",
|
||||
"id": "29817da1",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -728,7 +728,7 @@
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"id": "c2dc1c15",
|
||||
"id": "560fc9f4",
|
||||
"metadata": {
|
||||
"collapsed": false,
|
||||
"editable": true
|
||||
@@ -754,7 +754,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "81854928",
|
||||
"id": "3fc05383",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -779,7 +779,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "9a8a9dc4",
|
||||
"id": "bb21261e",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -793,7 +793,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "df82eef8",
|
||||
"id": "c5ab9dbf",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -823,7 +823,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "ba32d247",
|
||||
"id": "7d0dbb14",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
@@ -845,7 +845,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "cb0788db",
|
||||
"id": "e301049d",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
|
||||
@@ -55,8 +55,8 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
<script defer="defer" src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
<link rel="index" title="Index" href="genindex.html" />
|
||||
<link rel="search" title="Search" href="search.html" />
|
||||
<link rel="next" title="Project 1 on Machine Learning, deadline October 9 (midnight), 2023" href="project1.html" />
|
||||
<link rel="prev" title="Week 37: Statitsitcal interpretations and Resampling Methods" href="week37.html" />
|
||||
<link rel="next" title="Week 38: Logistic Regression and Optimization" href="week38.html" />
|
||||
<link rel="prev" title="Week 37: Statistical interpretations and Resampling Methods" href="week37.html" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<meta name="docsearch:language" content="None">
|
||||
|
||||
@@ -285,7 +285,7 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week37.html">
|
||||
Week 37: Statitsitcal interpretations and Resampling Methods
|
||||
Week 37: Statistical interpretations and Resampling Methods
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1 current active">
|
||||
@@ -293,6 +293,11 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
Exercises week 38
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week38.html">
|
||||
Week 38: Logistic Regression and Optimization
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
<p aria-level="2" class="caption" role="heading">
|
||||
<span class="caption-text">
|
||||
@@ -446,22 +451,24 @@ C(\boldsymbol{X},\boldsymbol{\beta}) =\frac{1}{n}\sum_{i=0}^{n-1}(y_i-\tilde{y}_
|
||||
\]</div>
|
||||
<p>Here the expected value <span class="math notranslate nohighlight">\(\mathbb{E}\)</span> is the sample value.</p>
|
||||
<p>Show that you can rewrite this in terms of a term which contains the variance of the model itself (the so-called variance term), a
|
||||
term which measures the deviation from the true data and the mean value of the model (the bias term) and finally the variance of the noise.
|
||||
That is, show that</p>
|
||||
term which measures the deviation from the true data and the mean value of the model (the bias term) and finally the variance of the noise.</p>
|
||||
<p>That is, show that</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\mathbb{E}\left[(\boldsymbol{y}-\boldsymbol{\tilde{y}})^2\right]=(\mathrm{Bias}[\tilde{y}])^2+\mathrm{var}[\tilde{f}]+\sigma^2,
|
||||
\mathbb{E}\left[(\boldsymbol{y}-\boldsymbol{\tilde{y}})^2\right]=\mathrm{Bias}[y]+\mathrm{var}[\tilde{y}]+\sigma^2,
|
||||
\]</div>
|
||||
<p>with</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
(\mathrm{Bias}[\tilde{y}])^2=\left(\boldsymbol{y}-\mathbb{E}\left[\boldsymbol{\tilde{y}}\right]\right)^2,
|
||||
\mathrm{Bias}[y]=\mathbb{E}\left[\left(\boldsymbol{y}-\mathbb{E}\left[\boldsymbol{\tilde{y}}\right]\right)^2\right],
|
||||
\]</div>
|
||||
<p>and</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\mathrm{var}[\tilde{f}]=\frac{1}{n}\sum_i(\tilde{y}_i-\mathbb{E}\left[\boldsymbol{\tilde{y}}\right])^2.
|
||||
\mathrm{var}[\tilde{y}]=\frac{1}{n}\sum_i(\tilde{y}_i-\mathbb{E}\left[\boldsymbol{\tilde{y}}\right])^2.
|
||||
\]</div>
|
||||
<p>The answer to this exercise should be included in the theory part of the report. This exercise is also part of the weekly exercises of week 37.
|
||||
Explain what the terms mean and discuss their interpretations.</p>
|
||||
<p>Explain what the terms mean and discuss their interpretations.</p>
|
||||
<p>Perform then a bias-variance analysis of a simple one-dimensional (or other models of your choice) function by
|
||||
studying the MSE value as function of the complexity of your model. Use ordinary least squares only.</p>
|
||||
@@ -502,13 +509,13 @@ You can follow the code example in the jupyter-book at <a class="reference exter
|
||||
<i class="fas fa-angle-left"></i>
|
||||
<div class="prev-next-info">
|
||||
<p class="prev-next-subtitle">previous</p>
|
||||
<p class="prev-next-title">Week 37: Statitsitcal interpretations and Resampling Methods</p>
|
||||
<p class="prev-next-title">Week 37: Statistical interpretations and Resampling Methods</p>
|
||||
</div>
|
||||
</a>
|
||||
<a class='right-next' id="next-link" href="project1.html" title="next page">
|
||||
<a class='right-next' id="next-link" href="week38.html" title="next page">
|
||||
<div class="prev-next-info">
|
||||
<p class="prev-next-subtitle">next</p>
|
||||
<p class="prev-next-title">Project 1 on Machine Learning, deadline October 9 (midnight), 2023</p>
|
||||
<p class="prev-next-title">Week 38: Logistic Regression and Optimization</p>
|
||||
</div>
|
||||
<i class="fas fa-angle-right"></i>
|
||||
</a>
|
||||
|
||||
@@ -55,7 +55,7 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
<script defer="defer" src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
<link rel="index" title="Index" href="genindex.html" />
|
||||
<link rel="search" title="Search" href="search.html" />
|
||||
<link rel="prev" title="Week 37: Statitsitcal interpretations and Resampling Methods" href="week37.html" />
|
||||
<link rel="prev" title="Week 38: Logistic Regression and Optimization" href="week38.html" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<meta name="docsearch:language" content="None">
|
||||
|
||||
@@ -284,7 +284,17 @@ const thebe_selector_output = ".output, .cell_output"
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week37.html">
|
||||
Week 37: Statitsitcal interpretations and Resampling Methods
|
||||
Week 37: Statistical interpretations and Resampling Methods
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="exercisesweek38.html">
|
||||
Exercises week 38
|
||||
</a>
|
||||
</li>
|
||||
<li class="toctree-l1">
|
||||
<a class="reference internal" href="week38.html">
|
||||
Week 38: Logistic Regression and Optimization
|
||||
</a>
|
||||
</li>
|
||||
</ul>
|
||||
@@ -649,7 +659,7 @@ which polynomial fits the data best.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_7740/39730396.py:11: MatplotlibDeprecationWarning: Calling gca() with keyword arguments was deprecated in Matplotlib 3.4. Starting two minor releases later, gca() will take no keyword arguments. The gca() function should only be used to get the current axes, or if no axes exist, create new axes with default keyword arguments. To create a new axes with non-default arguments, use plt.axes() or plt.subplot().
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20604/39730396.py:11: MatplotlibDeprecationWarning: Calling gca() with keyword arguments was deprecated in Matplotlib 3.4. Starting two minor releases later, gca() will take no keyword arguments. The gca() function should only be used to get the current axes, or if no axes exist, create new axes with default keyword arguments. To create a new axes with non-default arguments, use plt.axes() or plt.subplot().
|
||||
ax = fig.gca(projection='3d')
|
||||
</pre></div>
|
||||
</div>
|
||||
@@ -805,17 +815,17 @@ term which measures the deviation from the true data and the mean value of the m
|
||||
That is, show that</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\mathbb{E}\left[(\boldsymbol{y}-\boldsymbol{\tilde{y}})^2\right]=(\mathrm{Bias}[\tilde{y}])^2+\mathrm{var}[\tilde{f}]+\sigma^2,
|
||||
\mathbb{E}\left[(\boldsymbol{y}-\boldsymbol{\tilde{y}})^2\right]=\mathrm{Bias}[y]+\mathrm{var}[\tilde{y}]+\sigma^2,
|
||||
\]</div>
|
||||
<p>with</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
(\mathrm{Bias}[\tilde{y}])^2=\left(\boldsymbol{y}-\mathbb{E}\left[\boldsymbol{\tilde{y}}\right]\right)^2,
|
||||
\mathrm{Bias}[y]=\mathbb{E}\left[\left(\boldsymbol{y}-\mathbb{E}\left[\boldsymbol{\tilde{y}}\right]\right)^2\right],
|
||||
\]</div>
|
||||
<p>and</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\mathrm{var}[\tilde{f}]=\frac{1}{n}\sum_i(\tilde{y}_i-\mathbb{E}\left[\boldsymbol{\tilde{y}}\right])^2.
|
||||
\mathrm{var}[\tilde{y}]=\frac{1}{n}\sum_i(\tilde{y}_i-\mathbb{E}\left[\boldsymbol{\tilde{y}}\right])^2.
|
||||
\]</div>
|
||||
<p>The answer to this exercise should be included in the theory part of the report. This exercise is also part of the weekly exercises of week 37.
|
||||
Explain what the terms mean and discuss their interpretations.</p>
|
||||
@@ -1008,11 +1018,11 @@ of code developers and contributors keeps increasing.</p>
|
||||
|
||||
<!-- Previous / next buttons -->
|
||||
<div class='prev-next-area'>
|
||||
<a class='left-prev' id="prev-link" href="week37.html" title="previous page">
|
||||
<a class='left-prev' id="prev-link" href="week38.html" title="previous page">
|
||||
<i class="fas fa-angle-left"></i>
|
||||
<div class="prev-next-info">
|
||||
<p class="prev-next-subtitle">previous</p>
|
||||
<p class="prev-next-title">Week 37: Statitsitcal interpretations and Resampling Methods</p>
|
||||
<p class="prev-next-title">Week 38: Logistic Regression and Optimization</p>
|
||||
</div>
|
||||
</a>
|
||||
</div>
|
||||
|
||||
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user