lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
4f4c4e69464b1b4bc304ff84c9054e5ffdb7a4ee
FYS-STK4155/doc/pub/Regression/html/._Regression-bs068.html
T
mhjensen a4a41ce5d4 hints on project 2
2018-10-11 05:27:04 +02:00

494 lines
31 KiB
HTML
Raw Blame History

<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="description" content="Data Analysis and Machine Learning: Linear Regression and more Advanced Regression Analysis">
<title>Data Analysis and Machine Learning: Linear Regression and more Advanced Regression Analysis</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Regression analysis, overarching aims', 2, None, '___sec0'),
('Regression analysis, overarching aims II', 2, None, '___sec1'),
('General linear models', 2, None, '___sec2'),
('Rewriting the fitting procedure as a linear algebra problem',
2,
None,
'___sec3'),
('Rewriting the fitting procedure as a linear algebra problem, '
'follows',
2,
None,
'___sec4'),
('Generalizing the fitting procedure as a linear algebra problem',
2,
None,
'___sec5'),
('Generalizing the fitting procedure as a linear algebra problem',
2,
None,
'___sec6'),
('Optimizing our parameters', 2, None, '___sec7'),
('Optimizing our parameters, more details', 2, None, '___sec8'),
('Interpretations and optimizing our parameters',
2,
None,
'___sec9'),
('Interpretations and optimizing our parameters',
2,
None,
'___sec10'),
('Interpretations and optimizing our parameters',
2,
None,
'___sec11'),
('The $\\chi^2$ function', 2, None, '___sec12'),
('The $\\chi^2$ function', 2, None, '___sec13'),
('The $\\chi^2$ function', 2, None, '___sec14'),
('The $\\chi^2$ function', 2, None, '___sec15'),
('The $\\chi^2$ function', 2, None, '___sec16'),
('The $\\chi^2$ function', 2, None, '___sec17'),
('Simple regression model', 2, None, '___sec18'),
('Simple regression model, now using _scikit-learn_',
2,
None,
'___sec19'),
('Simple linear regression model using _scikit-learn_',
2,
None,
'___sec20'),
('Simple linear regression model', 2, None, '___sec21'),
('Less noise', 2, None, '___sec22'),
('How to study our fits', 2, None, '___sec23'),
('Minimizing the cost function', 2, None, '___sec24'),
('Relative error', 2, None, '___sec25'),
('The richness of _scikit-learn_', 2, None, '___sec26'),
('Functions in _scikit-learn_', 2, None, '___sec27'),
('Other functions in _scikit-learn_', 2, None, '___sec28'),
('The mean absolute error and other functions in _scikit-learn_',
2,
None,
'___sec29'),
('Cubic polynomial in _scikit-learn_', 2, None, '___sec30'),
('Polynomial Regression', 2, None, '___sec31'),
('Linking the regression analysis with a statistical '
'interpretation',
2,
None,
'___sec32'),
('Expectation value and variance', 2, None, '___sec33'),
('The singular value decompostion', 2, None, '___sec34'),
('From standard regression to Ridge regressions',
2,
None,
'___sec35'),
('Fixing the singularity', 2, None, '___sec36'),
('Fitting vs. predicting when data is in the model class',
2,
None,
'___sec37'),
('Fitting versus predicting when data is not in the model class',
2,
None,
'___sec38'),
('An example code without the model assessment part',
2,
None,
'___sec39'),
('Generating test data', 2, None, '___sec40'),
('How can we effectively evaluate the various models?',
2,
None,
'___sec41'),
('Code examples for Ridge and Lasso Regression',
2,
None,
'___sec42'),
('A second-order polynomial with Ridge and Lasso',
2,
None,
'___sec43'),
('Resampling methods', 2, None, '___sec44'),
('Resampling approaches can be computationally expensive',
2,
None,
'___sec45'),
('Why resampling methods ?', 2, None, '___sec46'),
('Statistical analysis', 2, None, '___sec47'),
('Statistics', 2, None, '___sec48'),
('Statistics, moments', 2, None, '___sec49'),
('Statistics, central moments', 2, None, '___sec50'),
('Statistics, covariance', 2, None, '___sec51'),
('Statistics, more covariance', 2, None, '___sec52'),
('Statistics, independent variables', 2, None, '___sec53'),
('Statistics, more variance', 2, None, '___sec54'),
('Statistics and stochastic processes', 2, None, '___sec55'),
('Statistics and sample variables', 2, None, '___sec56'),
('Statistics, sample variance and covariance',
2,
None,
'___sec57'),
('Statistics, law of large numbers', 2, None, '___sec58'),
('Statistics, more on sample error', 2, None, '___sec59'),
('Statistics', 2, None, '___sec60'),
('Statistics, central limit theorem', 2, None, '___sec61'),
('Statistics, more technicalities', 2, None, '___sec62'),
('Statistics', 2, None, '___sec63'),
('Statistics and sample variance', 2, None, '___sec64'),
('Statistics, uncorrelated results', 2, None, '___sec65'),
('Statistics, computations', 2, None, '___sec66'),
('Statistics, more on computations of errors',
2,
None,
'___sec67'),
('Statistics, wrapping up 1', 2, None, '___sec68'),
('Statistics, final expression', 2, None, '___sec69'),
('Statistics, effective number of correlations',
2,
None,
'___sec70'),
('Log-likelihood', 2, None, '___sec71'),
('Cross-validation', 2, None, '___sec72'),
('Computationally expensive', 2, None, '___sec73'),
('Various steps in cross-validation', 2, None, '___sec74'),
('How to set up the cross-validation for Ridge and/or Lasso',
2,
None,
'___sec75'),
('Predicted Residual Error Sum of Squares', 2, None, '___sec76'),
('Resampling methods: Jackknife and Bootstrap',
2,
None,
'___sec77'),
('Resampling methods: Jackknife', 2, None, '___sec78'),
('Resampling methods: Jackknife estimator', 2, None, '___sec79'),
('Jackknife code example', 2, None, '___sec80'),
('Resampling methods: Bootstrap', 2, None, '___sec81'),
('Resampling methods: Bootstrap background', 2, None, '___sec82'),
('Resampling methods: More Bootstrap background',
2,
None,
'___sec83'),
('Resampling methods: Bootstrap approach', 2, None, '___sec84'),
('Resampling methods: Bootstrap steps', 2, None, '___sec85'),
('Code example for the Bootstrap method', 2, None, '___sec86'),
('Resampling methods: Blocking', 2, None, '___sec87'),
('Blocking Transformations', 2, None, '___sec88'),
('Blocking Transformations', 2, None, '___sec89'),
('Blocking Transformations, getting there', 2, None, '___sec90'),
('Blocking Transformations, final expressions',
2,
None,
'___sec91'),
('"Code examples for Blocking, Jackknife and '
'bootstrap":"https://github.com/CompPhysics/MachineLearning/tree/master/doc/Programs/ResamplingAnalysisScripts"',
2,
None,
'___sec92'),
('The bias-variance tradeoff', 2, None, '___sec93'),
('Training and testing data', 2, None, '___sec94'),
('Procedure to find a predictor', 2, None, '___sec95'),
('What we want', 2, None, '___sec96'),
('The expected generalization error', 2, None, '___sec97'),
('Elaborating a little bit more', 2, None, '___sec98'),
('The bias', 2, None, '___sec99'),
('The variance', 2, None, '___sec100'),
('Summing up', 2, None, '___sec101'),
('The one-dimensional Ising model, project 2',
2,
None,
'___sec102'),
('Example: The one-dimensional Ising model',
2,
None,
'___sec103'),
('Reformulating the problem to suit regression',
2,
None,
'___sec104'),
('Linear regression', 2, None, '___sec105'),
('Ordinary least squares', 2, None, '___sec106'),
('Singular Value decomposition', 2, None, '___sec107'),
('Fitting with scikit-learn', 2, None, '___sec108'),
('Ridge regression', 2, None, '___sec109'),
('LASSO regression', 2, None, '___sec110'),
('Performance of the different models', 2, None, '___sec111'),
('Performance as function of the regularization parameter',
2,
None,
'___sec112'),
('Finding the optimal value of $\\lambda$',
2,
None,
'___sec113')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Regression-bs.html">Data Analysis and Machine Learning: Linear Regression and more Advanced Regression Analysis</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Regression-bs001.html#___sec0" style="font-size: 80%;">Regression analysis, overarching aims</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs002.html#___sec1" style="font-size: 80%;">Regression analysis, overarching aims II</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs003.html#___sec2" style="font-size: 80%;">General linear models</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs004.html#___sec3" style="font-size: 80%;">Rewriting the fitting procedure as a linear algebra problem</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs005.html#___sec4" style="font-size: 80%;">Rewriting the fitting procedure as a linear algebra problem, follows</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs006.html#___sec5" style="font-size: 80%;">Generalizing the fitting procedure as a linear algebra problem</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs007.html#___sec6" style="font-size: 80%;">Generalizing the fitting procedure as a linear algebra problem</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs008.html#___sec7" style="font-size: 80%;">Optimizing our parameters</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs009.html#___sec8" style="font-size: 80%;">Optimizing our parameters, more details</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs010.html#___sec9" style="font-size: 80%;">Interpretations and optimizing our parameters</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs011.html#___sec10" style="font-size: 80%;">Interpretations and optimizing our parameters</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs012.html#___sec11" style="font-size: 80%;">Interpretations and optimizing our parameters</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs013.html#___sec12" style="font-size: 80%;">The \( \chi^2 \) function</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs014.html#___sec13" style="font-size: 80%;">The \( \chi^2 \) function</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs015.html#___sec14" style="font-size: 80%;">The \( \chi^2 \) function</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs016.html#___sec15" style="font-size: 80%;">The \( \chi^2 \) function</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs017.html#___sec16" style="font-size: 80%;">The \( \chi^2 \) function</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs018.html#___sec17" style="font-size: 80%;">The \( \chi^2 \) function</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs019.html#___sec18" style="font-size: 80%;">Simple regression model</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs020.html#___sec19" style="font-size: 80%;">Simple regression model, now using <b>scikit-learn</b></a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs021.html#___sec20" style="font-size: 80%;">Simple linear regression model using <b>scikit-learn</b></a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs022.html#___sec21" style="font-size: 80%;">Simple linear regression model</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs023.html#___sec22" style="font-size: 80%;">Less noise</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs024.html#___sec23" style="font-size: 80%;">How to study our fits</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs025.html#___sec24" style="font-size: 80%;">Minimizing the cost function</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs026.html#___sec25" style="font-size: 80%;">Relative error</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs027.html#___sec26" style="font-size: 80%;">The richness of <b>scikit-learn</b></a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs028.html#___sec27" style="font-size: 80%;">Functions in <b>scikit-learn</b></a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs029.html#___sec28" style="font-size: 80%;">Other functions in <b>scikit-learn</b></a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs030.html#___sec29" style="font-size: 80%;">The mean absolute error and other functions in <b>scikit-learn</b></a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs031.html#___sec30" style="font-size: 80%;">Cubic polynomial in <b>scikit-learn</b></a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs032.html#___sec31" style="font-size: 80%;">Polynomial Regression</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs033.html#___sec32" style="font-size: 80%;">Linking the regression analysis with a statistical interpretation</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs034.html#___sec33" style="font-size: 80%;">Expectation value and variance</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs035.html#___sec34" style="font-size: 80%;">The singular value decompostion</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs036.html#___sec35" style="font-size: 80%;">From standard regression to Ridge regressions</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs037.html#___sec36" style="font-size: 80%;">Fixing the singularity</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs038.html#___sec37" style="font-size: 80%;">Fitting vs. predicting when data is in the model class</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs039.html#___sec38" style="font-size: 80%;">Fitting versus predicting when data is not in the model class</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs040.html#___sec39" style="font-size: 80%;">An example code without the model assessment part</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs041.html#___sec40" style="font-size: 80%;">Generating test data</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs042.html#___sec41" style="font-size: 80%;">How can we effectively evaluate the various models?</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs043.html#___sec42" style="font-size: 80%;">Code examples for Ridge and Lasso Regression</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs044.html#___sec43" style="font-size: 80%;">A second-order polynomial with Ridge and Lasso</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs045.html#___sec44" style="font-size: 80%;">Resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs046.html#___sec45" style="font-size: 80%;">Resampling approaches can be computationally expensive</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs047.html#___sec46" style="font-size: 80%;">Why resampling methods ?</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs048.html#___sec47" style="font-size: 80%;">Statistical analysis</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs049.html#___sec48" style="font-size: 80%;">Statistics</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs050.html#___sec49" style="font-size: 80%;">Statistics, moments</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs051.html#___sec50" style="font-size: 80%;">Statistics, central moments</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs052.html#___sec51" style="font-size: 80%;">Statistics, covariance</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs053.html#___sec52" style="font-size: 80%;">Statistics, more covariance</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs054.html#___sec53" style="font-size: 80%;">Statistics, independent variables</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs055.html#___sec54" style="font-size: 80%;">Statistics, more variance</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs056.html#___sec55" style="font-size: 80%;">Statistics and stochastic processes</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs057.html#___sec56" style="font-size: 80%;">Statistics and sample variables</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs058.html#___sec57" style="font-size: 80%;">Statistics, sample variance and covariance</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs059.html#___sec58" style="font-size: 80%;">Statistics, law of large numbers</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs060.html#___sec59" style="font-size: 80%;">Statistics, more on sample error</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs061.html#___sec60" style="font-size: 80%;">Statistics</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs062.html#___sec61" style="font-size: 80%;">Statistics, central limit theorem</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs063.html#___sec62" style="font-size: 80%;">Statistics, more technicalities</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs064.html#___sec63" style="font-size: 80%;">Statistics</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs065.html#___sec64" style="font-size: 80%;">Statistics and sample variance</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs066.html#___sec65" style="font-size: 80%;">Statistics, uncorrelated results</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs067.html#___sec66" style="font-size: 80%;">Statistics, computations</a></li>
<!-- navigation toc: --> <li><a href="#___sec67" style="font-size: 80%;">Statistics, more on computations of errors</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs069.html#___sec68" style="font-size: 80%;">Statistics, wrapping up 1</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs070.html#___sec69" style="font-size: 80%;">Statistics, final expression</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs071.html#___sec70" style="font-size: 80%;">Statistics, effective number of correlations</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs072.html#___sec71" style="font-size: 80%;">Log-likelihood</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs073.html#___sec72" style="font-size: 80%;">Cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs074.html#___sec73" style="font-size: 80%;">Computationally expensive</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs075.html#___sec74" style="font-size: 80%;">Various steps in cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs076.html#___sec75" style="font-size: 80%;">How to set up the cross-validation for Ridge and/or Lasso</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs077.html#___sec76" style="font-size: 80%;">Predicted Residual Error Sum of Squares</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs078.html#___sec77" style="font-size: 80%;">Resampling methods: Jackknife and Bootstrap</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs079.html#___sec78" style="font-size: 80%;">Resampling methods: Jackknife</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs080.html#___sec79" style="font-size: 80%;">Resampling methods: Jackknife estimator</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs081.html#___sec80" style="font-size: 80%;">Jackknife code example</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs082.html#___sec81" style="font-size: 80%;">Resampling methods: Bootstrap</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs083.html#___sec82" style="font-size: 80%;">Resampling methods: Bootstrap background</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs084.html#___sec83" style="font-size: 80%;">Resampling methods: More Bootstrap background</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs085.html#___sec84" style="font-size: 80%;">Resampling methods: Bootstrap approach</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs086.html#___sec85" style="font-size: 80%;">Resampling methods: Bootstrap steps</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs087.html#___sec86" style="font-size: 80%;">Code example for the Bootstrap method</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs088.html#___sec87" style="font-size: 80%;">Resampling methods: Blocking</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs089.html#___sec88" style="font-size: 80%;">Blocking Transformations</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs090.html#___sec89" style="font-size: 80%;">Blocking Transformations</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs091.html#___sec90" style="font-size: 80%;">Blocking Transformations, getting there</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs092.html#___sec91" style="font-size: 80%;">Blocking Transformations, final expressions</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs093.html#___sec92" style="font-size: 80%;">"Code examples for Blocking, Jackknife and bootstrap":"https://github.com/CompPhysics/MachineLearning/tree/master/doc/Programs/ResamplingAnalysisScripts"</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs094.html#___sec93" style="font-size: 80%;">The bias-variance tradeoff</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs095.html#___sec94" style="font-size: 80%;">Training and testing data</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs096.html#___sec95" style="font-size: 80%;">Procedure to find a predictor</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs097.html#___sec96" style="font-size: 80%;">What we want</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs098.html#___sec97" style="font-size: 80%;">The expected generalization error</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs099.html#___sec98" style="font-size: 80%;">Elaborating a little bit more</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs100.html#___sec99" style="font-size: 80%;">The bias</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs101.html#___sec100" style="font-size: 80%;">The variance</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs102.html#___sec101" style="font-size: 80%;">Summing up</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs103.html#___sec102" style="font-size: 80%;">The one-dimensional Ising model, project 2</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs104.html#___sec103" style="font-size: 80%;">Example: The one-dimensional Ising model</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs105.html#___sec104" style="font-size: 80%;">Reformulating the problem to suit regression</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs106.html#___sec105" style="font-size: 80%;">Linear regression</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs107.html#___sec106" style="font-size: 80%;">Ordinary least squares</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs108.html#___sec107" style="font-size: 80%;">Singular Value decomposition</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs109.html#___sec108" style="font-size: 80%;">Fitting with scikit-learn</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs110.html#___sec109" style="font-size: 80%;">Ridge regression</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs111.html#___sec110" style="font-size: 80%;">LASSO regression</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs112.html#___sec111" style="font-size: 80%;">Performance of the different models</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs113.html#___sec112" style="font-size: 80%;">Performance as function of the regularization parameter</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs114.html#___sec113" style="font-size: 80%;">Finding the optimal value of \( \lambda \)</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0068"></a>
<!-- !split -->
<h2 id="___sec67" class="anchor">Statistics, more on computations of errors </h2>
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
Computationally the uncorrelated first term is much easier to treat
efficiently than the second.
$$
\mathrm{var}(x) = \frac{1}{n}\sum_{k=1}^n (x_k - \bar x_n)^2 =
\left(\frac{1}{n}\sum_{k=1}^n x_k^2\right) - \bar x_n^2
$$
We just accumulate separately the values \( x^2 \) and \( x \) for every
measurement \( x \) we receive. The correlation term, though, has to be
calculated at the end of the experiment since we need all the
measurements to calculate the cross terms. Therefore, all measurements
have to be stored throughout the experiment.
</div>
</div>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Regression-bs067.html">&laquo;</a></li>
<li><a href="._Regression-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._Regression-bs060.html">61</a></li>
<li><a href="._Regression-bs061.html">62</a></li>
<li><a href="._Regression-bs062.html">63</a></li>
<li><a href="._Regression-bs063.html">64</a></li>
<li><a href="._Regression-bs064.html">65</a></li>
<li><a href="._Regression-bs065.html">66</a></li>
<li><a href="._Regression-bs066.html">67</a></li>
<li><a href="._Regression-bs067.html">68</a></li>
<li class="active"><a href="._Regression-bs068.html">69</a></li>
<li><a href="._Regression-bs069.html">70</a></li>
<li><a href="._Regression-bs070.html">71</a></li>
<li><a href="._Regression-bs071.html">72</a></li>
<li><a href="._Regression-bs072.html">73</a></li>
<li><a href="._Regression-bs073.html">74</a></li>
<li><a href="._Regression-bs074.html">75</a></li>
<li><a href="._Regression-bs075.html">76</a></li>
<li><a href="._Regression-bs076.html">77</a></li>
<li><a href="._Regression-bs077.html">78</a></li>
<li><a href="">...</a></li>
<li><a href="._Regression-bs114.html">115</a></li>
<li><a href="._Regression-bs069.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink