Files
FYS-STK4155/doc/pub/Regression/html/._Regression-bs066.html
T
2020-09-11 11:29:44 +02:00

457 lines
27 KiB
HTML

<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Linear Regression and more Advanced Regression Analysis">
<title>Data Analysis and Machine Learning: Linear Regression and more Advanced Regression Analysis</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Why Linear Regression (aka Ordinary Least Squares and family)',
2,
None,
'___sec0'),
('Regression analysis, overarching aims', 2, None, '___sec1'),
('Regression analysis, overarching aims II', 2, None, '___sec2'),
('Examples', 2, None, '___sec3'),
('General linear models', 2, None, '___sec4'),
('Rewriting the fitting procedure as a linear algebra problem',
2,
None,
'___sec5'),
('Rewriting the fitting procedure as a linear algebra problem, '
'more details',
2,
None,
'___sec6'),
('Generalizing the fitting procedure as a linear algebra problem',
2,
None,
'___sec7'),
('Generalizing the fitting procedure as a linear algebra problem',
2,
None,
'___sec8'),
('Optimizing our parameters', 2, None, '___sec9'),
('Our model for the nuclear binding energies',
2,
None,
'___sec10'),
('Optimizing our parameters, more details', 2, None, '___sec11'),
('Interpretations and optimizing our parameters',
2,
None,
'___sec12'),
('Interpretations and optimizing our parameters',
2,
None,
'___sec13'),
('Some useful matrix and vector expressions',
2,
None,
'___sec14'),
('Interpretations and optimizing our parameters',
2,
None,
'___sec15'),
('Own code for Ordinary Least Squares', 2, None, '___sec16'),
('Adding error analysis and training set up',
2,
None,
'___sec17'),
('The $\\chi^2$ function', 2, None, '___sec18'),
('The $\\chi^2$ function', 2, None, '___sec19'),
('The $\\chi^2$ function', 2, None, '___sec20'),
('The $\\chi^2$ function', 2, None, '___sec21'),
('The $\\chi^2$ function', 2, None, '___sec22'),
('The $\\chi^2$ function', 2, None, '___sec23'),
('Fitting an Equation of State for Dense Nuclear Matter',
2,
None,
'___sec24'),
('The code', 2, None, '___sec25'),
('Splitting our Data in Training and Test data',
2,
None,
'___sec26'),
('The Boston housing data example', 2, None, '___sec27'),
('Housing data, the code', 2, None, '___sec28'),
('Reducing the number of degrees of freedom, overarching view',
2,
None,
'___sec29'),
('Preprocessing our data', 2, None, '___sec30'),
('More preprocessing', 2, None, '___sec31'),
('Simple preprocessing examples, Franke function and regression',
2,
None,
'___sec32'),
('The singular value decomposition', 2, None, '___sec33'),
('Linear Regression Problems', 2, None, '___sec34'),
('Fixing the singularity', 2, None, '___sec35'),
('Basic math of the SVD', 2, None, '___sec36'),
('The SVD, a Fantastic Algorithm', 2, None, '___sec37'),
('Economy-size SVD', 2, None, '___sec38'),
('Codes for the SVD', 2, None, '___sec39'),
('Mathematical Properties', 2, None, '___sec40'),
('Ridge and LASSO Regression', 2, None, '___sec41'),
('More on Ridge Regression', 2, None, '___sec42'),
('Interpreting the Ridge results', 2, None, '___sec43'),
('More interpretations', 2, None, '___sec44'),
('A better understanding of regularization', 2, None, '___sec45'),
('Decomposing the OLS and Ridge expressions',
2,
None,
'___sec46'),
('Introducing the Covariance and Correlation functions',
2,
None,
'___sec47'),
('Correlation Function and Design/Feature Matrix',
2,
None,
'___sec48'),
('Covariance Matrix Examples', 2, None, '___sec49'),
('Correlation Matrix', 2, None, '___sec50'),
('Correlation Matrix with Pandas', 2, None, '___sec51'),
('Correlation Matrix with Pandas and the Franke function',
2,
None,
'___sec52'),
('Rewriting the Covariance and/or Correlation Matrix',
2,
None,
'___sec53'),
('Linking with SVD', 2, None, '___sec54'),
('Where are we going?', 2, None, '___sec55'),
('Resampling methods', 2, None, '___sec56'),
('Resampling approaches can be computationally expensive',
2,
None,
'___sec57'),
('Why resampling methods ?', 2, None, '___sec58'),
('Statistical analysis', 2, None, '___sec59'),
('Linking the regression analysis with a statistical '
'interpretation',
2,
None,
'___sec60'),
('Assumptions made', 2, None, '___sec61'),
('Expectation value and variance', 2, None, '___sec62'),
('Expectation value and variance for $\\boldsymbol{\\beta}$',
2,
None,
'___sec63'),
('Resampling methods', 2, None, '___sec64'),
('Resampling methods: Jackknife and Bootstrap',
2,
None,
'___sec65'),
('Resampling methods: Jackknife', 2, None, '___sec66'),
('Jackknife code example', 2, None, '___sec67'),
('Resampling methods: Bootstrap', 2, None, '___sec68'),
('Resampling methods: Bootstrap background', 2, None, '___sec69'),
('Resampling methods: More Bootstrap background',
2,
None,
'___sec70'),
('Resampling methods: Bootstrap approach', 2, None, '___sec71'),
('Resampling methods: Bootstrap steps', 2, None, '___sec72'),
('Code example for the Bootstrap method', 2, None, '___sec73'),
('Various steps in cross-validation', 2, None, '___sec74'),
('How to set up the cross-validation for Ridge and/or Lasso',
2,
None,
'___sec75'),
('Cross-validation in brief', 2, None, '___sec76'),
('Code Example for Cross-validation and $k$-fold '
'Cross-validation',
2,
None,
'___sec77'),
('The bias-variance tradeoff', 2, None, '___sec78'),
('Example code for Bias-Variance tradeoff', 2, None, '___sec79'),
('Understanding what happens', 2, None, '___sec80'),
('Summing up', 2, None, '___sec81'),
("Another Example from Scikit-Learn's Repository",
2,
None,
'___sec82'),
('More examples on bootstrap and cross-validation and errors',
2,
None,
'___sec83'),
('The same example but now with cross-validation',
2,
None,
'___sec84'),
('Cross-validation with Ridge', 2, None, '___sec85'),
('The Ising model', 2, None, '___sec86'),
('Reformulating the problem to suit regression',
2,
None,
'___sec87'),
('Linear regression', 2, None, '___sec88'),
('Singular Value decomposition', 2, None, '___sec89'),
('The one-dimensional Ising model', 2, None, '___sec90'),
('Ridge regression', 2, None, '___sec91'),
('LASSO regression', 2, None, '___sec92'),
('Performance as function of the regularization parameter',
2,
None,
'___sec93'),
('Finding the optimal value of $\\lambda$', 2, None, '___sec94')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Regression-bs.html">Data Analysis and Machine Learning: Linear Regression and more Advanced Regression Analysis</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Regression-bs001.html#___sec0" style="font-size: 80%;">Why Linear Regression (aka Ordinary Least Squares and family)</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs002.html#___sec1" style="font-size: 80%;">Regression analysis, overarching aims</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs003.html#___sec2" style="font-size: 80%;">Regression analysis, overarching aims II</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs004.html#___sec3" style="font-size: 80%;">Examples</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs005.html#___sec4" style="font-size: 80%;">General linear models</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs006.html#___sec5" style="font-size: 80%;">Rewriting the fitting procedure as a linear algebra problem</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs007.html#___sec6" style="font-size: 80%;">Rewriting the fitting procedure as a linear algebra problem, more details</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs008.html#___sec7" style="font-size: 80%;">Generalizing the fitting procedure as a linear algebra problem</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs009.html#___sec8" style="font-size: 80%;">Generalizing the fitting procedure as a linear algebra problem</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs010.html#___sec9" style="font-size: 80%;">Optimizing our parameters</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs011.html#___sec10" style="font-size: 80%;">Our model for the nuclear binding energies</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs012.html#___sec11" style="font-size: 80%;">Optimizing our parameters, more details</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs013.html#___sec12" style="font-size: 80%;">Interpretations and optimizing our parameters</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs014.html#___sec13" style="font-size: 80%;">Interpretations and optimizing our parameters</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs015.html#___sec14" style="font-size: 80%;">Some useful matrix and vector expressions</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs016.html#___sec15" style="font-size: 80%;">Interpretations and optimizing our parameters</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs017.html#___sec16" style="font-size: 80%;">Own code for Ordinary Least Squares</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs018.html#___sec17" style="font-size: 80%;">Adding error analysis and training set up</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs019.html#___sec18" style="font-size: 80%;">The \( \chi^2 \) function</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs020.html#___sec19" style="font-size: 80%;">The \( \chi^2 \) function</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs021.html#___sec20" style="font-size: 80%;">The \( \chi^2 \) function</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs022.html#___sec21" style="font-size: 80%;">The \( \chi^2 \) function</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs023.html#___sec22" style="font-size: 80%;">The \( \chi^2 \) function</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs024.html#___sec23" style="font-size: 80%;">The \( \chi^2 \) function</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs025.html#___sec24" style="font-size: 80%;">Fitting an Equation of State for Dense Nuclear Matter</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs026.html#___sec25" style="font-size: 80%;">The code</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs027.html#___sec26" style="font-size: 80%;">Splitting our Data in Training and Test data</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs028.html#___sec27" style="font-size: 80%;">The Boston housing data example</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs029.html#___sec28" style="font-size: 80%;">Housing data, the code</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs030.html#___sec29" style="font-size: 80%;">Reducing the number of degrees of freedom, overarching view</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs031.html#___sec30" style="font-size: 80%;">Preprocessing our data</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs032.html#___sec31" style="font-size: 80%;">More preprocessing</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs033.html#___sec32" style="font-size: 80%;">Simple preprocessing examples, Franke function and regression</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs034.html#___sec33" style="font-size: 80%;">The singular value decomposition</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs035.html#___sec34" style="font-size: 80%;">Linear Regression Problems</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs036.html#___sec35" style="font-size: 80%;">Fixing the singularity</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs037.html#___sec36" style="font-size: 80%;">Basic math of the SVD</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs038.html#___sec37" style="font-size: 80%;">The SVD, a Fantastic Algorithm</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs039.html#___sec38" style="font-size: 80%;">Economy-size SVD</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs040.html#___sec39" style="font-size: 80%;">Codes for the SVD</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs041.html#___sec40" style="font-size: 80%;">Mathematical Properties</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs042.html#___sec41" style="font-size: 80%;">Ridge and LASSO Regression</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs043.html#___sec42" style="font-size: 80%;">More on Ridge Regression</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs044.html#___sec43" style="font-size: 80%;">Interpreting the Ridge results</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs045.html#___sec44" style="font-size: 80%;">More interpretations</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs046.html#___sec45" style="font-size: 80%;">A better understanding of regularization</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs047.html#___sec46" style="font-size: 80%;">Decomposing the OLS and Ridge expressions</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs048.html#___sec47" style="font-size: 80%;">Introducing the Covariance and Correlation functions</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs049.html#___sec48" style="font-size: 80%;">Correlation Function and Design/Feature Matrix</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs050.html#___sec49" style="font-size: 80%;">Covariance Matrix Examples</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs051.html#___sec50" style="font-size: 80%;">Correlation Matrix</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs052.html#___sec51" style="font-size: 80%;">Correlation Matrix with Pandas</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs053.html#___sec52" style="font-size: 80%;">Correlation Matrix with Pandas and the Franke function</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs054.html#___sec53" style="font-size: 80%;">Rewriting the Covariance and/or Correlation Matrix</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs055.html#___sec54" style="font-size: 80%;">Linking with SVD</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs056.html#___sec55" style="font-size: 80%;">Where are we going?</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs057.html#___sec56" style="font-size: 80%;">Resampling methods</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs058.html#___sec57" style="font-size: 80%;">Resampling approaches can be computationally expensive</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs059.html#___sec58" style="font-size: 80%;">Why resampling methods ?</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs060.html#___sec59" style="font-size: 80%;">Statistical analysis</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs061.html#___sec60" style="font-size: 80%;">Linking the regression analysis with a statistical interpretation</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs062.html#___sec61" style="font-size: 80%;">Assumptions made</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs063.html#___sec62" style="font-size: 80%;">Expectation value and variance</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs064.html#___sec63" style="font-size: 80%;">Expectation value and variance for \( \boldsymbol{\beta} \)</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs065.html#___sec64" style="font-size: 80%;">Resampling methods</a></li>
<!-- navigation toc: --> <li><a href="#___sec65" style="font-size: 80%;">Resampling methods: Jackknife and Bootstrap</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs067.html#___sec66" style="font-size: 80%;">Resampling methods: Jackknife</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs068.html#___sec67" style="font-size: 80%;">Jackknife code example</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs069.html#___sec68" style="font-size: 80%;">Resampling methods: Bootstrap</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs070.html#___sec69" style="font-size: 80%;">Resampling methods: Bootstrap background</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs071.html#___sec70" style="font-size: 80%;">Resampling methods: More Bootstrap background</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs072.html#___sec71" style="font-size: 80%;">Resampling methods: Bootstrap approach</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs073.html#___sec72" style="font-size: 80%;">Resampling methods: Bootstrap steps</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs074.html#___sec73" style="font-size: 80%;">Code example for the Bootstrap method</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs075.html#___sec74" style="font-size: 80%;">Various steps in cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs076.html#___sec75" style="font-size: 80%;">How to set up the cross-validation for Ridge and/or Lasso</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs077.html#___sec76" style="font-size: 80%;">Cross-validation in brief</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs078.html#___sec77" style="font-size: 80%;">Code Example for Cross-validation and \( k \)-fold Cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs079.html#___sec78" style="font-size: 80%;">The bias-variance tradeoff</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs080.html#___sec79" style="font-size: 80%;">Example code for Bias-Variance tradeoff</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs081.html#___sec80" style="font-size: 80%;">Understanding what happens</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs082.html#___sec81" style="font-size: 80%;">Summing up</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs083.html#___sec82" style="font-size: 80%;">Another Example from Scikit-Learn's Repository</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs084.html#___sec83" style="font-size: 80%;">More examples on bootstrap and cross-validation and errors</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs085.html#___sec84" style="font-size: 80%;">The same example but now with cross-validation</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs086.html#___sec85" style="font-size: 80%;">Cross-validation with Ridge</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs087.html#___sec86" style="font-size: 80%;">The Ising model</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs088.html#___sec87" style="font-size: 80%;">Reformulating the problem to suit regression</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs089.html#___sec88" style="font-size: 80%;">Linear regression</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs090.html#___sec89" style="font-size: 80%;">Singular Value decomposition</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs091.html#___sec90" style="font-size: 80%;">The one-dimensional Ising model</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs092.html#___sec91" style="font-size: 80%;">Ridge regression</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs093.html#___sec92" style="font-size: 80%;">LASSO regression</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs094.html#___sec93" style="font-size: 80%;">Performance as function of the regularization parameter</a></li>
<!-- navigation toc: --> <li><a href="._Regression-bs095.html#___sec94" style="font-size: 80%;">Finding the optimal value of \( \lambda \)</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0066"></a>
<!-- !split -->
<h2 id="___sec65" class="anchor">Resampling methods: Jackknife and Bootstrap </h2>
<p>
Two famous
resampling methods are the <b>independent bootstrap</b> and <b>the jackknife</b>.
<p>
The jackknife is a special case of the independent bootstrap. Still, the jackknife was made
popular prior to the independent bootstrap. And as the popularity of
the independent bootstrap soared, new variants, such as <b>the dependent bootstrap</b>.
<p>
The Jackknife and independent bootstrap work for
independent, identically distributed random variables.
If these conditions are not
satisfied, the methods will fail. Yet, it should be said that if the data are
independent, identically distributed, and we only want to estimate the
variance of \( \overline{X} \) (which often is the case), then there is no
need for bootstrapping.
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Regression-bs065.html">&laquo;</a></li>
<li><a href="._Regression-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._Regression-bs058.html">59</a></li>
<li><a href="._Regression-bs059.html">60</a></li>
<li><a href="._Regression-bs060.html">61</a></li>
<li><a href="._Regression-bs061.html">62</a></li>
<li><a href="._Regression-bs062.html">63</a></li>
<li><a href="._Regression-bs063.html">64</a></li>
<li><a href="._Regression-bs064.html">65</a></li>
<li><a href="._Regression-bs065.html">66</a></li>
<li class="active"><a href="._Regression-bs066.html">67</a></li>
<li><a href="._Regression-bs067.html">68</a></li>
<li><a href="._Regression-bs068.html">69</a></li>
<li><a href="._Regression-bs069.html">70</a></li>
<li><a href="._Regression-bs070.html">71</a></li>
<li><a href="._Regression-bs071.html">72</a></li>
<li><a href="._Regression-bs072.html">73</a></li>
<li><a href="._Regression-bs073.html">74</a></li>
<li><a href="._Regression-bs074.html">75</a></li>
<li><a href="._Regression-bs075.html">76</a></li>
<li><a href="">...</a></li>
<li><a href="._Regression-bs095.html">96</a></li>
<li><a href="._Regression-bs067.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>