lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
5c6c05efbe058f033f3f35def695f64a7c70b9ce
FYS-STK4155/doc/pub/Regression/html/._Regression-bs065.html
T
mhjensen d0b8f0f458 typos in reg slides
2020-09-11 11:29:44 +02:00

463 lines
28 KiB
HTML
Raw Blame History

<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Linear Regression and more Advanced Regression Analysis">
<title>Data Analysis and Machine Learning: Linear Regression and more Advanced Regression Analysis</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Why Linear Regression (aka Ordinary Least Squares and family)',
2,
None,
'___sec0'),
('Regression analysis, overarching aims', 2, None, '___sec1'),
('Regression analysis, overarching aims II', 2, None, '___sec2'),
('Examples', 2, None, '___sec3'),
('General linear models', 2, None, '___sec4'),
('Rewriting the fitting procedure as a linear algebra problem',
2,
None,
'___sec5'),
('Rewriting the fitting procedure as a linear algebra problem, '
'more details',
2,
None,
'___sec6'),
('Generalizing the fitting procedure as a linear algebra problem',
2,
None,
'___sec7'),
('Generalizing the fitting procedure as a linear algebra problem',
2,
None,
'___sec8'),
('Optimizing our parameters', 2, None, '___sec9'),
('Our model for the nuclear binding energies',
2,
None,
'___sec10'),
('Optimizing our parameters, more details', 2, None, '___sec11'),
('Interpretations and optimizing our parameters',
2,
None,
'___sec12'),
('Interpretations and optimizing our parameters',
2,
None,
'___sec13'),
('Some useful matrix and vector expressions',
2,
None,
'___sec14'),
('Interpretations and optimizing our parameters',
2,
None,
'___sec15'),
('Own code for Ordinary Least Squares', 2, None, '___sec16'),
('Adding error analysis and training set up',
2,
None,
'___sec17'),
('The $\\chi^2$ function', 2, None, '___sec18'),
('The $\\chi^2$ function', 2, None, '___sec19'),
('The $\\chi^2$ function', 2, None, '___sec20'),
('The $\\chi^2$ function', 2, None, '___sec21'),
('The $\\chi^2$ function', 2, None, '___sec22'),
('The $\\chi^2$ function', 2, None, '___sec23'),
('Fitting an Equation of State for Dense Nuclear Matter',
2,
None,
'___sec24'),
('The code', 2, None, '___sec25'),
('Splitting our Data in Training and Test data',
2,
None,
'___sec26'),
('The Boston housing data example', 2, None, '___sec27'),
('Housing data, the code', 2, None, '___sec28'),
('Reducing the number of degrees of freedom, overarching view',
2,
None,
'___sec29'),
('Preprocessing our data', 2, None, '___sec30'),
('More preprocessing', 2, None, '___sec31'),
('Simple preprocessing examples, Franke function and regression',
2,
None,
'___sec32'),
('The singular value decomposition', 2, None, '___sec33'),
('Linear Regression Problems', 2, None, '___sec34'),
('Fixing the singularity', 2, None, '___sec35'),
('Basic math of the SVD', 2, None, '___sec36'),
('The SVD, a Fantastic Algorithm', 2, None, '___sec37'),
('Economy-size SVD', 2, None, '___sec38'),
('Codes for the SVD', 2, None, '___sec39'),
('Mathematical Properties', 2, None, '___sec40'),
('Ridge and LASSO Regression', 2, None, '___sec41'),
('More on Ridge Regression', 2, None, '___sec42'),
('Interpreting the Ridge results', 2, None, '___sec43'),
('More interpretations', 2, None, '___sec44'),
('A better understanding of regularization', 2, None, '___sec45'),
('Decomposing the OLS and Ridge expressions',
2,
None,
'___sec46'),
('Introducing the Covariance and Correlation functions',
2,
None,
'___sec47'),
('Correlation Function and Design/Feature Matrix',
2,
None,
'___sec48'),
('Covariance Matrix Examples', 2, None, '___sec49'),
('Correlation Matrix', 2, None, '___sec50'),
('Correlation Matrix with Pandas', 2, None, '___sec51'),
('Correlation Matrix with Pandas and the Franke function',
2,
None,
'___sec52'),
('Rewriting the Covariance and/or Correlation Matrix',
2,
None,
'___sec53'),
('Linking with SVD', 2, None, '___sec54'),
('Where are we going?', 2, None, '___sec55'),
('Resampling methods', 2, None, '___sec56'),
('Resampling approaches can be computationally expensive',
2,
None,
'___sec57'),
('Why resampling methods ?', 2, None, '___sec58'),
('Statistical analysis', 2, None, '___sec59'),
('Linking the regression analysis with a statistical '
'interpretation',
2,
None,
'___sec60'),
('Assumptions made', 2, None, '___sec61'),
('Expectation value and variance', 2, None, '___sec62'),
('Expectation value and variance for $\\boldsymbol{\\beta}$',
2,
None,
'___sec63'),
('Resampling methods', 2, None, '___sec64'),
('Resampling methods: Jackknife and Bootstrap',
2,
None,
'___sec65'),
('Resampling methods: Jackknife', 2, None, '___sec66'),
('Jackknife code example', 2, None, '___sec67'),
('Resampling methods: Bootstrap', 2, None, '___sec68'),
('Resampling methods: Bootstrap background', 2, None, '___sec69'),
('Resampling methods: More Bootstrap background',
2,
None,
'___sec70'),
('Resampling methods: Bootstrap approach', 2, None, '___sec71'),
('Resampling methods: Bootstrap steps', 2, None, '___sec72'),
('Code example for the Bootstrap method', 2, None, '___sec73'),
('Various steps in cross-validation', 2, None, '___sec74'),
('How to set up the cross-validation for Ridge and/or Lasso',
2,
None,
'___sec75'),
('Cross-validation in brief', 2, None, '___sec76'),
('Code Example for Cross-validation and $k$-fold '
'Cross-validation',
2,
None,
'___sec77'),
('The bias-variance tradeoff', 2, None, '___sec78'),
('Example code for Bias-Variance tradeoff', 2, None, '___sec79'),
('Understanding what happens', 2, None, '___sec80'),
('Summing up', 2, None, '___sec81'),
("Another Example from Scikit-Learn's Repository",
2,
None,
'___sec82'),
('More examples on bootstrap and cross-validation and errors',
2,
None,
'___sec83'),
('The same example but now with cross-validation',
2,
None,
'___sec84'),
('Cross-validation with Ridge', 2, None, '___sec85'),
('The Ising model', 2, None, '___sec86'),
('Reformulating the problem to suit regression',
2,
None,
'___sec87'),
('Linear regression', 2, None, '___sec88'),
('Singular Value decomposition', 2, None, '___sec89'),
('The one-dimensional Ising model', 2, None, '___sec90'),
('Ridge regression', 2, None, '___sec91'),
('LASSO regression', 2, None, '___sec92'),
('Performance as function of the regularization parameter',
2,
None,
'___sec93'),
('Finding the optimal value of $\\lambda$', 2, None, '___sec94')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Regression-bs.html">Data Analysis and Machine Learning: Linear Regression and more Advanced Regression Analysis</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Regression-bs001.html#___sec0" style="font-size: 80%;">Why Linear Regression (aka Ordinary Least Squares and family)</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs002.html#___sec1" style="font-size: 80%;">Regression analysis, overarching aims</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs003.html#___sec2" style="font-size: 80%;">Regression analysis, overarching aims II</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs004.html#___sec3" style="font-size: 80%;">Examples</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs005.html#___sec4" style="font-size: 80%;">General linear models</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs006.html#___sec5" style="font-size: 80%;">Rewriting the fitting procedure as a linear algebra problem</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs007.html#___sec6" style="font-size: 80%;">Rewriting the fitting procedure as a linear algebra problem, more details</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs008.html#___sec7" style="font-size: 80%;">Generalizing the fitting procedure as a linear algebra problem</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs009.html#___sec8" style="font-size: 80%;">Generalizing the fitting procedure as a linear algebra problem</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs010.html#___sec9" style="font-size: 80%;">Optimizing our parameters</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs011.html#___sec10" style="font-size: 80%;">Our model for the nuclear binding energies</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs012.html#___sec11" style="font-size: 80%;">Optimizing our parameters, more details</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs013.html#___sec12" style="font-size: 80%;">Interpretations and optimizing our parameters</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs014.html#___sec13" style="font-size: 80%;">Interpretations and optimizing our parameters</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs015.html#___sec14" style="font-size: 80%;">Some useful matrix and vector expressions</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs016.html#___sec15" style="font-size: 80%;">Interpretations and optimizing our parameters</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs017.html#___sec16" style="font-size: 80%;">Own code for Ordinary Least Squares</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs018.html#___sec17" style="font-size: 80%;">Adding error analysis and training set up</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs019.html#___sec18" style="font-size: 80%;">The \( \chi^2 \) function</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs020.html#___sec19" style="font-size: 80%;">The \( \chi^2 \) function</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs021.html#___sec20" style="font-size: 80%;">The \( \chi^2 \) function</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs022.html#___sec21" style="font-size: 80%;">The \( \chi^2 \) function</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs023.html#___sec22" style="font-size: 80%;">The \( \chi^2 \) function</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs024.html#___sec23" style="font-size: 80%;">The \( \chi^2 \) function</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs025.html#___sec24" style="font-size: 80%;">Fitting an Equation of State for Dense Nuclear Matter</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs026.html#___sec25" style="font-size: 80%;">The code</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs027.html#___sec26" style="font-size: 80%;">Splitting our Data in Training and Test data</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs028.html#___sec27" style="font-size: 80%;">The Boston housing data example</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs029.html#___sec28" style="font-size: 80%;">Housing data, the code</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs030.html#___sec29" style="font-size: 80%;">Reducing the number of degrees of freedom, overarching view</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs031.html#___sec30" style="font-size: 80%;">Preprocessing our data</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs032.html#___sec31" style="font-size: 80%;">More preprocessing</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs033.html#___sec32" style="font-size: 80%;">Simple preprocessing examples, Franke function and regression</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs034.html#___sec33" style="font-size: 80%;">The singular value decomposition</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs035.html#___sec34" style="font-size: 80%;">Linear Regression Problems</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs036.html#___sec35" style="font-size: 80%;">Fixing the singularity</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs037.html#___sec36" style="font-size: 80%;">Basic math of the SVD</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs038.html#___sec37" style="font-size: 80%;">The SVD, a Fantastic Algorithm</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs039.html#___sec38" style="font-size: 80%;">Economy-size SVD</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs040.html#___sec39" style="font-size: 80%;">Codes for the SVD</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs041.html#___sec40" style="font-size: 80%;">Mathematical Properties</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs042.html#___sec41" style="font-size: 80%;">Ridge and LASSO Regression</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs043.html#___sec42" style="font-size: 80%;">More on Ridge Regression</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs044.html#___sec43" style="font-size: 80%;">Interpreting the Ridge results</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs045.html#___sec44" style="font-size: 80%;">More interpretations</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs046.html#___sec45" style="font-size: 80%;">A better understanding of regularization</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs047.html#___sec46" style="font-size: 80%;">Decomposing the OLS and Ridge expressions</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs048.html#___sec47" style="font-size: 80%;">Introducing the Covariance and Correlation functions</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs049.html#___sec48" style="font-size: 80%;">Correlation Function and Design/Feature Matrix</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs050.html#___sec49" style="font-size: 80%;">Covariance Matrix Examples</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs051.html#___sec50" style="font-size: 80%;">Correlation Matrix</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs052.html#___sec51" style="font-size: 80%;">Correlation Matrix with Pandas</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs053.html#___sec52" style="font-size: 80%;">Correlation Matrix with Pandas and the Franke function</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs054.html#___sec53" style="font-size: 80%;">Rewriting the Covariance and/or Correlation Matrix</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs055.html#___sec54" style="font-size: 80%;">Linking with SVD</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs056.html#___sec55" style="font-size: 80%;">Where are we going?</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs057.html#___sec56" style="font-size: 80%;">Resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs058.html#___sec57" style="font-size: 80%;">Resampling approaches can be computationally expensive</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs059.html#___sec58" style="font-size: 80%;">Why resampling methods ?</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs060.html#___sec59" style="font-size: 80%;">Statistical analysis</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs061.html#___sec60" style="font-size: 80%;">Linking the regression analysis with a statistical interpretation</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs062.html#___sec61" style="font-size: 80%;">Assumptions made</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs063.html#___sec62" style="font-size: 80%;">Expectation value and variance</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs064.html#___sec63" style="font-size: 80%;">Expectation value and variance for \( \boldsymbol{\beta} \)</a></li>
<!-- navigation toc: --> <li><a href="#___sec64" style="font-size: 80%;">Resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs066.html#___sec65" style="font-size: 80%;">Resampling methods: Jackknife and Bootstrap</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs067.html#___sec66" style="font-size: 80%;">Resampling methods: Jackknife</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs068.html#___sec67" style="font-size: 80%;">Jackknife code example</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs069.html#___sec68" style="font-size: 80%;">Resampling methods: Bootstrap</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs070.html#___sec69" style="font-size: 80%;">Resampling methods: Bootstrap background</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs071.html#___sec70" style="font-size: 80%;">Resampling methods: More Bootstrap background</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs072.html#___sec71" style="font-size: 80%;">Resampling methods: Bootstrap approach</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs073.html#___sec72" style="font-size: 80%;">Resampling methods: Bootstrap steps</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs074.html#___sec73" style="font-size: 80%;">Code example for the Bootstrap method</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs075.html#___sec74" style="font-size: 80%;">Various steps in cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs076.html#___sec75" style="font-size: 80%;">How to set up the cross-validation for Ridge and/or Lasso</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs077.html#___sec76" style="font-size: 80%;">Cross-validation in brief</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs078.html#___sec77" style="font-size: 80%;">Code Example for Cross-validation and \( k \)-fold Cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs079.html#___sec78" style="font-size: 80%;">The bias-variance tradeoff</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs080.html#___sec79" style="font-size: 80%;">Example code for Bias-Variance tradeoff</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs081.html#___sec80" style="font-size: 80%;">Understanding what happens</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs082.html#___sec81" style="font-size: 80%;">Summing up</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs083.html#___sec82" style="font-size: 80%;">Another Example from Scikit-Learn's Repository</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs084.html#___sec83" style="font-size: 80%;">More examples on bootstrap and cross-validation and errors</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs085.html#___sec84" style="font-size: 80%;">The same example but now with cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs086.html#___sec85" style="font-size: 80%;">Cross-validation with Ridge</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs087.html#___sec86" style="font-size: 80%;">The Ising model</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs088.html#___sec87" style="font-size: 80%;">Reformulating the problem to suit regression</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs089.html#___sec88" style="font-size: 80%;">Linear regression</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs090.html#___sec89" style="font-size: 80%;">Singular Value decomposition</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs091.html#___sec90" style="font-size: 80%;">The one-dimensional Ising model</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs092.html#___sec91" style="font-size: 80%;">Ridge regression</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs093.html#___sec92" style="font-size: 80%;">LASSO regression</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs094.html#___sec93" style="font-size: 80%;">Performance as function of the regularization parameter</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs095.html#___sec94" style="font-size: 80%;">Finding the optimal value of \( \lambda \)</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0065"></a>
<!-- !split -->
<h2 id="___sec64" class="anchor">Resampling methods </h2>
<p>
With all these analytical equations for both the OLS and Ridge
regression, we will now outline how to assess a given model. This will
lead us to a discussion of the so-called bias-variance tradeoff (see
below) and so-called resampling methods.
<p>
One of the quantities we have discussed as a way to measure errors is
the mean-squared error (MSE), mainly used for fitting of continuous
functions. Another choice is the absolute error.
<p>
In the discussions below we will focus on the MSE and in particular since we will split the data into test and training data,
we discuss the
<ol>
<li> prediction error or simply the <b>test error</b> \( \mathrm{Err_{Test}} \), where we have a fixed training set and the test error is the MSE arising from the data reserved for testing. We discuss also the</li>
<li> training error \( \mathrm{Err_{Train}} \), which is the average loss over the training data.</li>
</ol>
As our model becomes more and more complex, more of the training data tends to used. The training may thence adapt to more complicated structures in the data. This may lead to a decrease in the bias (see below for code example) and a slight increase of the variance for the test error.
For a certain level of complexity the test error will reach minimum, before starting to increase again. The
training error reaches a saturation.
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Regression-bs064.html">&laquo;</a></li>
<li><a href="._Regression-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._Regression-bs057.html">58</a></li>
<li><a href="._Regression-bs058.html">59</a></li>
<li><a href="._Regression-bs059.html">60</a></li>
<li><a href="._Regression-bs060.html">61</a></li>
<li><a href="._Regression-bs061.html">62</a></li>
<li><a href="._Regression-bs062.html">63</a></li>
<li><a href="._Regression-bs063.html">64</a></li>
<li><a href="._Regression-bs064.html">65</a></li>
<li class="active"><a href="._Regression-bs065.html">66</a></li>
<li><a href="._Regression-bs066.html">67</a></li>
<li><a href="._Regression-bs067.html">68</a></li>
<li><a href="._Regression-bs068.html">69</a></li>
<li><a href="._Regression-bs069.html">70</a></li>
<li><a href="._Regression-bs070.html">71</a></li>
<li><a href="._Regression-bs071.html">72</a></li>
<li><a href="._Regression-bs072.html">73</a></li>
<li><a href="._Regression-bs073.html">74</a></li>
<li><a href="._Regression-bs074.html">75</a></li>
<li><a href="">...</a></li>
<li><a href="._Regression-bs095.html">96</a></li>
<li><a href="._Regression-bs066.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink