update on files
This commit is contained in:
mhjensen
@@ -165,7 +165,7 @@ MathJax.Hub.Config({
|
||||
<center><b>Department of Physics, University of Oslo, Norway</b></center>
|
||||
<br>
|
||||
<p>
|
||||
<center><h4>Sep 7, 2020</h4></center> <!-- date -->
|
||||
<center><h4>Sep 16, 2020</h4></center> <!-- date -->
|
||||
<br>
|
||||
<p>
|
||||
</div> <!-- end jumbotron -->
|
||||
@@ -404,18 +404,16 @@ Note also that when you calculate the bias, in all applications you don't know t
|
||||
The aim here is to write your own code for another widely popular
|
||||
resampling technique, the so-called cross-validation method. Again,
|
||||
before you start with cross-validation approach, you should scale your
|
||||
data and split it in test and training data as you did earlier.
|
||||
Perform a resampling of the data where you split the data in training
|
||||
data and test data using for example
|
||||
data.
|
||||
|
||||
<p>
|
||||
Implement the \( k \)-fold cross-validation algorithm (write your own
|
||||
code) and evaluate again the MSE function resulting
|
||||
from the test data. You can compare your own code with that from
|
||||
from the test folds. You can compare your own code with that from
|
||||
<b>Scikit-Learn</b> if needed.
|
||||
|
||||
<p>
|
||||
Compare the MSE you get from your cross-validation code with the one you got from your <b>bootstrap</b> code.
|
||||
Compare the MSE you get from your cross-validation code with the one you got from your <b>bootstrap</b> code. Comment your results. Try \( 5-10 \) folds.
|
||||
You can also compare your own cross-validation code with the one provided by <b>Scikit-Learn</b>.
|
||||
|
||||
<h3 id="___sec4" class="anchor">Part d): Ridge Regression on the Franke function with resampling </h3>
|
||||
|
||||
@@ -165,7 +165,7 @@ MathJax.Hub.Config({
|
||||
<center><b>Department of Physics, University of Oslo, Norway</b></center>
|
||||
<br>
|
||||
<p>
|
||||
<center><h4>Sep 7, 2020</h4></center> <!-- date -->
|
||||
<center><h4>Sep 16, 2020</h4></center> <!-- date -->
|
||||
<br>
|
||||
<p>
|
||||
</div> <!-- end jumbotron -->
|
||||
@@ -404,18 +404,16 @@ Note also that when you calculate the bias, in all applications you don't know t
|
||||
The aim here is to write your own code for another widely popular
|
||||
resampling technique, the so-called cross-validation method. Again,
|
||||
before you start with cross-validation approach, you should scale your
|
||||
data and split it in test and training data as you did earlier.
|
||||
Perform a resampling of the data where you split the data in training
|
||||
data and test data using for example
|
||||
data.
|
||||
|
||||
<p>
|
||||
Implement the \( k \)-fold cross-validation algorithm (write your own
|
||||
code) and evaluate again the MSE function resulting
|
||||
from the test data. You can compare your own code with that from
|
||||
from the test folds. You can compare your own code with that from
|
||||
<b>Scikit-Learn</b> if needed.
|
||||
|
||||
<p>
|
||||
Compare the MSE you get from your cross-validation code with the one you got from your <b>bootstrap</b> code.
|
||||
Compare the MSE you get from your cross-validation code with the one you got from your <b>bootstrap</b> code. Comment your results. Try \( 5-10 \) folds.
|
||||
You can also compare your own cross-validation code with the one provided by <b>Scikit-Learn</b>.
|
||||
|
||||
<h3 id="___sec4" class="anchor">Part d): Ridge Regression on the Franke function with resampling </h3>
|
||||
|
||||
@@ -122,7 +122,7 @@ MathJax.Hub.Config({
|
||||
<center><b>Department of Physics, University of Oslo, Norway</b></center>
|
||||
<br>
|
||||
<p>
|
||||
<center><h4>Sep 7, 2020</h4></center> <!-- date -->
|
||||
<center><h4>Sep 16, 2020</h4></center> <!-- date -->
|
||||
<br>
|
||||
|
||||
<h2 id="___sec0">Regression analysis and resampling methods </h2>
|
||||
@@ -359,18 +359,16 @@ Note also that when you calculate the bias, in all applications you don't know t
|
||||
The aim here is to write your own code for another widely popular
|
||||
resampling technique, the so-called cross-validation method. Again,
|
||||
before you start with cross-validation approach, you should scale your
|
||||
data and split it in test and training data as you did earlier.
|
||||
Perform a resampling of the data where you split the data in training
|
||||
data and test data using for example
|
||||
data.
|
||||
|
||||
<p>
|
||||
Implement the \( k \)-fold cross-validation algorithm (write your own
|
||||
code) and evaluate again the MSE function resulting
|
||||
from the test data. You can compare your own code with that from
|
||||
from the test folds. You can compare your own code with that from
|
||||
<b>Scikit-Learn</b> if needed.
|
||||
|
||||
<p>
|
||||
Compare the MSE you get from your cross-validation code with the one you got from your <b>bootstrap</b> code.
|
||||
Compare the MSE you get from your cross-validation code with the one you got from your <b>bootstrap</b> code. Comment your results. Try \( 5-10 \) folds.
|
||||
You can also compare your own cross-validation code with the one provided by <b>Scikit-Learn</b>.
|
||||
|
||||
<h3 id="___sec4">Part d): Ridge Regression on the Franke function with resampling </h3>
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
"<!-- Author: --> \n",
|
||||
"**[Data Analysis and Machine Learning FYS-STK3155/FYS4155](http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html)**, Department of Physics, University of Oslo, Norway\n",
|
||||
"\n",
|
||||
"Date: **Sep 7, 2020**\n",
|
||||
"Date: **Sep 16, 2020**\n",
|
||||
"\n",
|
||||
"Copyright 1999-2020, [Data Analysis and Machine Learning FYS-STK3155/FYS4155](http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html). Released under CC Attribution-NonCommercial 4.0 license\n",
|
||||
"\n",
|
||||
@@ -318,19 +318,15 @@
|
||||
"The aim here is to write your own code for another widely popular\n",
|
||||
"resampling technique, the so-called cross-validation method. Again,\n",
|
||||
"before you start with cross-validation approach, you should scale your\n",
|
||||
"data and split it in test and training data as you did earlier.\n",
|
||||
"Perform a resampling of the data where you split the data in training\n",
|
||||
"data and test data using for example\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"data.\n",
|
||||
"\n",
|
||||
"Implement the $k$-fold cross-validation algorithm (write your own\n",
|
||||
"code) and evaluate again the MSE function resulting\n",
|
||||
"from the test data. You can compare your own code with that from\n",
|
||||
"from the test folds. You can compare your own code with that from\n",
|
||||
"**Scikit-Learn** if needed. \n",
|
||||
"\n",
|
||||
"Compare the MSE you get from your cross-validation code with the one you got from your **bootstrap** code.\n",
|
||||
"You can also compare your own cross-validation code with the one provided by **Scikit-Learn**.\n",
|
||||
"Compare the MSE you get from your cross-validation code with the one you got from your **bootstrap** code. Comment your results. Try $5-10$ folds. \n",
|
||||
"You can also compare your own cross-validation code with the one provided by **Scikit-Learn**. \n",
|
||||
"\n",
|
||||
"\n",
|
||||
"### Part d): Ridge Regression on the Franke function with resampling\n",
|
||||
|
||||
Binary file not shown.
@@ -0,0 +1,25 @@
|
||||
\relax
|
||||
\providecommand\hyper@newdestlabel[2]{}
|
||||
\providecommand\HyperFirstAtBeginDocument{\AtBeginDocument}
|
||||
\HyperFirstAtBeginDocument{\ifx\hyper@anchor\@undefined
|
||||
\global\let\oldcontentsline\contentsline
|
||||
\gdef\contentsline#1#2#3#4{\oldcontentsline{#1}{#2}{#3}}
|
||||
\global\let\oldnewlabel\newlabel
|
||||
\gdef\newlabel#1#2{\newlabelxx{#1}#2}
|
||||
\gdef\newlabelxx#1#2#3#4#5#6{\oldnewlabel{#1}{{#2}{#3}}}
|
||||
\AtEndDocument{\ifx\hyper@anchor\@undefined
|
||||
\let\contentsline\oldcontentsline
|
||||
\let\newlabel\oldnewlabel
|
||||
\fi}
|
||||
\fi}
|
||||
\global\let\hyper@last\relax
|
||||
\gdef\HyperFirstAtBeginDocument#1{#1}
|
||||
\providecommand\HyField@AuxAddToFields[1]{}
|
||||
\providecommand\HyField@AuxAddToCoFields[2]{}
|
||||
\@writefile{toc}{\contentsline {paragraph}{Part a): Ordinary Least Square (OLS) on the Franke function.}{3}{section*.2}}
|
||||
\@writefile{toc}{\contentsline {paragraph}{Part b): Bias-variance trade-off and resamplng techniques.}{4}{section*.3}}
|
||||
\@writefile{toc}{\contentsline {paragraph}{Part c) Cross-validation as resampling techniques, adding more complexity.}{5}{section*.4}}
|
||||
\@writefile{toc}{\contentsline {paragraph}{Part d): Ridge Regression on the Franke function with resampling.}{5}{section*.5}}
|
||||
\@writefile{toc}{\contentsline {paragraph}{Part e): Lasso Regression on the Franke function with resampling.}{5}{section*.6}}
|
||||
\@writefile{toc}{\contentsline {paragraph}{Part f): Introducing real data and preparing the data analysis.}{5}{section*.7}}
|
||||
\@writefile{toc}{\contentsline {paragraph}{Part g) OLS, Ridge and Lasso regression with resampling.}{6}{section*.8}}
|
||||
@@ -0,0 +1,851 @@
|
||||
This is pdfTeX, Version 3.14159265-2.6-1.40.16 (TeX Live 2015/Debian) (preloaded format=pdflatex 2018.10.20) 16 SEP 2020 17:14
|
||||
entering extended mode
|
||||
restricted \write18 enabled.
|
||||
%&-line parsing enabled.
|
||||
**Project1
|
||||
(./Project1.tex
|
||||
LaTeX2e <2016/02/01>
|
||||
Babel <3.9q> and hyphenation patterns for 81 language(s) loaded.
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/base/article.cls
|
||||
Document Class: article 2014/09/29 v1.4h Standard LaTeX document class
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/base/size10.clo
|
||||
File: size10.clo 2014/09/29 v1.4h Standard LaTeX file (size option)
|
||||
)
|
||||
\c@part=\count79
|
||||
\c@section=\count80
|
||||
\c@subsection=\count81
|
||||
\c@subsubsection=\count82
|
||||
\c@paragraph=\count83
|
||||
\c@subparagraph=\count84
|
||||
\c@figure=\count85
|
||||
\c@table=\count86
|
||||
\abovecaptionskip=\skip41
|
||||
\belowcaptionskip=\skip42
|
||||
\bibindent=\dimen102
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/relsize/relsize.sty
|
||||
Package: relsize 2013/03/29 ver 4.1
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/base/makeidx.sty
|
||||
Package: makeidx 2014/09/29 v1.0m Standard LaTeX package
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/graphics/color.sty
|
||||
Package: color 2016/01/03 v1.1b Standard LaTeX Color (DPC)
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/latexconfig/color.cfg
|
||||
File: color.cfg 2007/01/18 v1.5 color configuration of teTeX/TeXLive
|
||||
)
|
||||
Package color Info: Driver file: pdftex.def on input line 143.
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/pdftex-def/pdftex.def
|
||||
File: pdftex.def 2011/05/27 v0.06d Graphics/color for pdfTeX
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/generic/oberdiek/infwarerr.sty
|
||||
Package: infwarerr 2010/04/08 v1.3 Providing info/warning/error messages (HO)
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/generic/oberdiek/ltxcmds.sty
|
||||
Package: ltxcmds 2011/11/09 v1.22 LaTeX kernel commands for general use (HO)
|
||||
)
|
||||
\Gread@gobject=\count87
|
||||
))
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/setspace/setspace.sty
|
||||
Package: setspace 2011/12/19 v6.7a set line spacing
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsmath.sty
|
||||
Package: amsmath 2016/03/03 v2.15a AMS math features
|
||||
\@mathmargin=\skip43
|
||||
|
||||
For additional information on amsmath, use the `?' option.
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amstext.sty
|
||||
Package: amstext 2000/06/29 v2.01 AMS text
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsgen.sty
|
||||
File: amsgen.sty 1999/11/30 v2.0 generic functions
|
||||
\@emptytoks=\toks14
|
||||
\ex@=\dimen103
|
||||
))
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsbsy.sty
|
||||
Package: amsbsy 1999/11/29 v1.2d Bold Symbols
|
||||
\pmbraise@=\dimen104
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/amsmath/amsopn.sty
|
||||
Package: amsopn 1999/12/14 v2.01 operator names
|
||||
)
|
||||
\inf@bad=\count88
|
||||
LaTeX Info: Redefining \frac on input line 199.
|
||||
\uproot@=\count89
|
||||
\leftroot@=\count90
|
||||
LaTeX Info: Redefining \overline on input line 297.
|
||||
\classnum@=\count91
|
||||
\DOTSCASE@=\count92
|
||||
LaTeX Info: Redefining \ldots on input line 394.
|
||||
LaTeX Info: Redefining \dots on input line 397.
|
||||
LaTeX Info: Redefining \cdots on input line 518.
|
||||
\Mathstrutbox@=\box26
|
||||
\strutbox@=\box27
|
||||
\big@size=\dimen105
|
||||
LaTeX Font Info: Redeclaring font encoding OML on input line 630.
|
||||
LaTeX Font Info: Redeclaring font encoding OMS on input line 631.
|
||||
\macc@depth=\count93
|
||||
\c@MaxMatrixCols=\count94
|
||||
\dotsspace@=\muskip10
|
||||
\c@parentequation=\count95
|
||||
\dspbrk@lvl=\count96
|
||||
\tag@help=\toks15
|
||||
\row@=\count97
|
||||
\column@=\count98
|
||||
\maxfields@=\count99
|
||||
\andhelp@=\toks16
|
||||
\eqnshift@=\dimen106
|
||||
\alignsep@=\dimen107
|
||||
\tagshift@=\dimen108
|
||||
\tagwidth@=\dimen109
|
||||
\totwidth@=\dimen110
|
||||
\lineht@=\dimen111
|
||||
\@envbody=\toks17
|
||||
\multlinegap=\skip44
|
||||
\multlinetaggap=\skip45
|
||||
\mathdisplay@stack=\toks18
|
||||
LaTeX Info: Redefining \[ on input line 2735.
|
||||
LaTeX Info: Redefining \] on input line 2736.
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/amsfonts/amsfonts.sty
|
||||
Package: amsfonts 2013/01/14 v3.01 Basic AMSFonts support
|
||||
\symAMSa=\mathgroup4
|
||||
\symAMSb=\mathgroup5
|
||||
LaTeX Font Info: Overwriting math alphabet `\mathfrak' in version `bold'
|
||||
(Font) U/euf/m/n --> U/euf/b/n on input line 106.
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/amsfonts/amssymb.sty
|
||||
Package: amssymb 2013/01/14 v3.01 AMS font symbols
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/xcolor/xcolor.sty
|
||||
Package: xcolor 2007/01/21 v2.11 LaTeX color extensions (UK)
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/latexconfig/color.cfg
|
||||
File: color.cfg 2007/01/18 v1.5 color configuration of teTeX/TeXLive
|
||||
)
|
||||
Package xcolor Info: Driver file: pdftex.def on input line 225.
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/colortbl/colortbl.sty
|
||||
Package: colortbl 2012/02/13 v1.0a Color table columns (DPC)
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/tools/array.sty
|
||||
Package: array 2014/10/28 v2.4c Tabular extension package (FMi)
|
||||
\col@sep=\dimen112
|
||||
\extrarowheight=\dimen113
|
||||
\NC@list=\toks19
|
||||
\extratabsurround=\skip46
|
||||
\backup@length=\skip47
|
||||
)
|
||||
\everycr=\toks20
|
||||
\minrowclearance=\skip48
|
||||
)
|
||||
LaTeX Info: Redefining \color on input line 702.
|
||||
\rownum=\count100
|
||||
Package xcolor Info: Model `cmy' substituted by `cmy0' on input line 1337.
|
||||
Package xcolor Info: Model `hsb' substituted by `rgb' on input line 1341.
|
||||
Package xcolor Info: Model `RGB' extended on input line 1353.
|
||||
Package xcolor Info: Model `HTML' substituted by `rgb' on input line 1355.
|
||||
Package xcolor Info: Model `Hsb' substituted by `hsb' on input line 1356.
|
||||
Package xcolor Info: Model `tHsb' substituted by `hsb' on input line 1357.
|
||||
Package xcolor Info: Model `HSB' substituted by `hsb' on input line 1358.
|
||||
Package xcolor Info: Model `Gray' substituted by `gray' on input line 1359.
|
||||
Package xcolor Info: Model `wave' substituted by `hsb' on input line 1360.
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/tools/bm.sty
|
||||
Package: bm 2014/10/28 v1.1c Bold Symbol Support (DPC/FMi)
|
||||
\symboldoperators=\mathgroup6
|
||||
\symboldletters=\mathgroup7
|
||||
\symboldsymbols=\mathgroup8
|
||||
LaTeX Font Info: Redeclaring math alphabet \mathbf on input line 141.
|
||||
LaTeX Info: Redefining \bm on input line 207.
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/ltablex/ltablex.sty
|
||||
Package: ltablex 2014/08/13 v1.1 Modified tabularx
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/tools/longtable.sty
|
||||
Package: longtable 2014/10/28 v4.11 Multi-page Table package (DPC)
|
||||
\LTleft=\skip49
|
||||
\LTright=\skip50
|
||||
\LTpre=\skip51
|
||||
\LTpost=\skip52
|
||||
\LTchunksize=\count101
|
||||
\LTcapwidth=\dimen114
|
||||
\LT@head=\box28
|
||||
\LT@firsthead=\box29
|
||||
\LT@foot=\box30
|
||||
\LT@lastfoot=\box31
|
||||
\LT@cols=\count102
|
||||
\LT@rows=\count103
|
||||
\c@LT@tables=\count104
|
||||
\c@LT@chunks=\count105
|
||||
\LT@p@ftn=\toks21
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/tools/tabularx.sty
|
||||
Package: tabularx 2014/10/28 v2.10 `tabularx' package (DPC)
|
||||
\TX@col@width=\dimen115
|
||||
\TX@old@table=\dimen116
|
||||
\TX@old@col=\dimen117
|
||||
\TX@target=\dimen118
|
||||
\TX@delta=\dimen119
|
||||
\TX@cols=\count106
|
||||
\TX@ftn=\toks22
|
||||
))
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/microtype/microtype.sty
|
||||
Package: microtype 2013/05/23 v2.5a Micro-typographical refinements (RS)
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/graphics/keyval.sty
|
||||
Package: keyval 2014/10/28 v1.15 key=value parser (DPC)
|
||||
\KV@toks@=\toks23
|
||||
)
|
||||
\MT@toks=\toks24
|
||||
\MT@count=\count107
|
||||
LaTeX Info: Redefining \textls on input line 766.
|
||||
\MT@outer@kern=\dimen120
|
||||
LaTeX Info: Redefining \textmicrotypecontext on input line 1285.
|
||||
\MT@listname@count=\count108
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/microtype/microtype-pdftex.def
|
||||
File: microtype-pdftex.def 2013/05/23 v2.5a Definitions specific to pdftex (RS)
|
||||
|
||||
LaTeX Info: Redefining \lsstyle on input line 915.
|
||||
LaTeX Info: Redefining \lslig on input line 915.
|
||||
\MT@outer@space=\skip53
|
||||
)
|
||||
Package microtype Info: Loading configuration file microtype.cfg.
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/microtype/microtype.cfg
|
||||
File: microtype.cfg 2013/05/23 v2.5a microtype main configuration file (RS)
|
||||
))
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/graphics/graphicx.sty
|
||||
Package: graphicx 2014/10/28 v1.0g Enhanced LaTeX Graphics (DPC,SPQR)
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/graphics/graphics.sty
|
||||
Package: graphics 2016/01/03 v1.0q Standard LaTeX Graphics (DPC,SPQR)
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/graphics/trig.sty
|
||||
Package: trig 2016/01/03 v1.10 sin cos tan (DPC)
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/latexconfig/graphics.cfg
|
||||
File: graphics.cfg 2010/04/23 v1.9 graphics configuration of TeX Live
|
||||
)
|
||||
Package graphics Info: Driver file: pdftex.def on input line 95.
|
||||
)
|
||||
\Gin@req@height=\dimen121
|
||||
\Gin@req@width=\dimen122
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/fancyvrb/fancyvrb.sty
|
||||
Package: fancyvrb 2008/02/07
|
||||
|
||||
Style option: `fancyvrb' v2.7a, with DG/SPQR fixes, and firstline=lastline fix
|
||||
<2008/02/07> (tvz)
|
||||
\FV@CodeLineNo=\count109
|
||||
\FV@InFile=\read1
|
||||
\FV@TabBox=\box32
|
||||
\c@FancyVerbLine=\count110
|
||||
\FV@StepNumber=\count111
|
||||
\FV@OutFile=\write3
|
||||
) (/usr/share/texlive/texmf-dist/tex/latex/base/fontenc.sty
|
||||
Package: fontenc 2005/09/27 v1.99g Standard LaTeX package
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/base/t1enc.def
|
||||
File: t1enc.def 2005/09/27 v1.99g Standard LaTeX file
|
||||
LaTeX Font Info: Redeclaring font encoding T1 on input line 48.
|
||||
))
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/ucs/ucs.sty
|
||||
Package: ucs 2013/05/11 v2.2 UCS: Unicode input support
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/ucs/data/uni-global.def
|
||||
File: uni-global.def 2013/05/13 UCS: Unicode global data
|
||||
)
|
||||
\uc@secondtry=\count112
|
||||
\uc@combtoks=\toks25
|
||||
\uc@combtoksb=\toks26
|
||||
\uc@temptokena=\toks27
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/base/inputenc.sty
|
||||
Package: inputenc 2015/03/17 v1.2c Input encoding file
|
||||
\inpenc@prehook=\toks28
|
||||
\inpenc@posthook=\toks29
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/ucs/utf8x.def
|
||||
File: utf8x.def 2004/10/17 UCS: Input encoding UTF-8
|
||||
))
|
||||
(/usr/share/texmf/tex/latex/lm/lmodern.sty
|
||||
Package: lmodern 2009/10/30 v1.6 Latin Modern Fonts
|
||||
LaTeX Font Info: Overwriting symbol font `operators' in version `normal'
|
||||
(Font) OT1/cmr/m/n --> OT1/lmr/m/n on input line 22.
|
||||
LaTeX Font Info: Overwriting symbol font `letters' in version `normal'
|
||||
(Font) OML/cmm/m/it --> OML/lmm/m/it on input line 23.
|
||||
LaTeX Font Info: Overwriting symbol font `symbols' in version `normal'
|
||||
(Font) OMS/cmsy/m/n --> OMS/lmsy/m/n on input line 24.
|
||||
LaTeX Font Info: Overwriting symbol font `largesymbols' in version `normal'
|
||||
(Font) OMX/cmex/m/n --> OMX/lmex/m/n on input line 25.
|
||||
LaTeX Font Info: Overwriting symbol font `operators' in version `bold'
|
||||
(Font) OT1/cmr/bx/n --> OT1/lmr/bx/n on input line 26.
|
||||
LaTeX Font Info: Overwriting symbol font `letters' in version `bold'
|
||||
(Font) OML/cmm/b/it --> OML/lmm/b/it on input line 27.
|
||||
LaTeX Font Info: Overwriting symbol font `symbols' in version `bold'
|
||||
(Font) OMS/cmsy/b/n --> OMS/lmsy/b/n on input line 28.
|
||||
LaTeX Font Info: Overwriting symbol font `largesymbols' in version `bold'
|
||||
(Font) OMX/cmex/m/n --> OMX/lmex/m/n on input line 29.
|
||||
LaTeX Font Info: Overwriting math alphabet `\mathbf' in version `normal'
|
||||
(Font) OT1/cmr/bx/n --> OT1/lmr/bx/n on input line 31.
|
||||
LaTeX Font Info: Overwriting math alphabet `\mathsf' in version `normal'
|
||||
(Font) OT1/cmss/m/n --> OT1/lmss/m/n on input line 32.
|
||||
LaTeX Font Info: Overwriting math alphabet `\mathit' in version `normal'
|
||||
(Font) OT1/cmr/m/it --> OT1/lmr/m/it on input line 33.
|
||||
LaTeX Font Info: Overwriting math alphabet `\mathtt' in version `normal'
|
||||
(Font) OT1/cmtt/m/n --> OT1/lmtt/m/n on input line 34.
|
||||
LaTeX Font Info: Overwriting math alphabet `\mathbf' in version `bold'
|
||||
(Font) OT1/cmr/bx/n --> OT1/lmr/bx/n on input line 35.
|
||||
LaTeX Font Info: Overwriting math alphabet `\mathsf' in version `bold'
|
||||
(Font) OT1/cmss/bx/n --> OT1/lmss/bx/n on input line 36.
|
||||
LaTeX Font Info: Overwriting math alphabet `\mathit' in version `bold'
|
||||
(Font) OT1/cmr/bx/it --> OT1/lmr/bx/it on input line 37.
|
||||
LaTeX Font Info: Overwriting math alphabet `\mathtt' in version `bold'
|
||||
(Font) OT1/cmtt/m/n --> OT1/lmtt/m/n on input line 38.
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/hyperref.sty
|
||||
Package: hyperref 2012/11/06 v6.83m Hypertext links for LaTeX
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/generic/oberdiek/hobsub-hyperref.sty
|
||||
Package: hobsub-hyperref 2012/05/28 v1.13 Bundle oberdiek, subset hyperref (HO)
|
||||
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/generic/oberdiek/hobsub-generic.sty
|
||||
Package: hobsub-generic 2012/05/28 v1.13 Bundle oberdiek, subset generic (HO)
|
||||
Package: hobsub 2012/05/28 v1.13 Construct package bundles (HO)
|
||||
Package hobsub Info: Skipping package `infwarerr' (already loaded).
|
||||
Package hobsub Info: Skipping package `ltxcmds' (already loaded).
|
||||
Package: ifluatex 2010/03/01 v1.3 Provides the ifluatex switch (HO)
|
||||
Package ifluatex Info: LuaTeX not detected.
|
||||
Package: ifvtex 2010/03/01 v1.5 Detect VTeX and its facilities (HO)
|
||||
Package ifvtex Info: VTeX not detected.
|
||||
Package: intcalc 2007/09/27 v1.1 Expandable calculations with integers (HO)
|
||||
Package: ifpdf 2011/01/30 v2.3 Provides the ifpdf switch (HO)
|
||||
Package ifpdf Info: pdfTeX in PDF mode is detected.
|
||||
Package: etexcmds 2011/02/16 v1.5 Avoid name clashes with e-TeX commands (HO)
|
||||
Package etexcmds Info: Could not find \expanded.
|
||||
(etexcmds) That can mean that you are not using pdfTeX 1.50 or
|
||||
(etexcmds) that some package has redefined \expanded.
|
||||
(etexcmds) In the latter case, load this package earlier.
|
||||
Package: kvsetkeys 2012/04/25 v1.16 Key value parser (HO)
|
||||
Package: kvdefinekeys 2011/04/07 v1.3 Define keys (HO)
|
||||
Package: pdftexcmds 2011/11/29 v0.20 Utility functions of pdfTeX for LuaTeX (HO
|
||||
)
|
||||
Package pdftexcmds Info: LuaTeX not detected.
|
||||
Package pdftexcmds Info: \pdf@primitive is available.
|
||||
Package pdftexcmds Info: \pdf@ifprimitive is available.
|
||||
Package pdftexcmds Info: \pdfdraftmode found.
|
||||
Package: pdfescape 2011/11/25 v1.13 Implements pdfTeX's escape features (HO)
|
||||
Package: bigintcalc 2012/04/08 v1.3 Expandable calculations on big integers (HO
|
||||
)
|
||||
Package: bitset 2011/01/30 v1.1 Handle bit-vector datatype (HO)
|
||||
Package: uniquecounter 2011/01/30 v1.2 Provide unlimited unique counter (HO)
|
||||
)
|
||||
Package hobsub Info: Skipping package `hobsub' (already loaded).
|
||||
Package: letltxmacro 2010/09/02 v1.4 Let assignment for LaTeX macros (HO)
|
||||
Package: hopatch 2012/05/28 v1.2 Wrapper for package hooks (HO)
|
||||
Package: xcolor-patch 2011/01/30 xcolor patch
|
||||
Package: atveryend 2011/06/30 v1.8 Hooks at the very end of document (HO)
|
||||
Package atveryend Info: \enddocument detected (standard20110627).
|
||||
Package: atbegshi 2011/10/05 v1.16 At begin shipout hook (HO)
|
||||
Package: refcount 2011/10/16 v3.4 Data extraction from label references (HO)
|
||||
Package: hycolor 2011/01/30 v1.7 Color options for hyperref/bookmark (HO)
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/generic/ifxetex/ifxetex.sty
|
||||
Package: ifxetex 2010/09/12 v0.6 Provides ifxetex conditional
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/oberdiek/auxhook.sty
|
||||
Package: auxhook 2011/03/04 v1.3 Hooks for auxiliary files (HO)
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/oberdiek/kvoptions.sty
|
||||
Package: kvoptions 2011/06/30 v3.11 Key value format for package options (HO)
|
||||
)
|
||||
\@linkdim=\dimen123
|
||||
\Hy@linkcounter=\count113
|
||||
\Hy@pagecounter=\count114
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/pd1enc.def
|
||||
File: pd1enc.def 2012/11/06 v6.83m Hyperref: PDFDocEncoding definition (HO)
|
||||
)
|
||||
\Hy@SavedSpaceFactor=\count115
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/latexconfig/hyperref.cfg
|
||||
File: hyperref.cfg 2002/06/06 v1.2 hyperref configuration of TeXLive
|
||||
)
|
||||
Package hyperref Info: Option `final' set `true' on input line 4319.
|
||||
Package hyperref Info: Hyper figures OFF on input line 4443.
|
||||
Package hyperref Info: Link nesting OFF on input line 4448.
|
||||
Package hyperref Info: Hyper index ON on input line 4451.
|
||||
Package hyperref Info: Plain pages OFF on input line 4458.
|
||||
Package hyperref Info: Backreferencing OFF on input line 4463.
|
||||
Package hyperref Info: Implicit mode ON; LaTeX internals redefined.
|
||||
Package hyperref Info: Bookmarks ON on input line 4688.
|
||||
\c@Hy@tempcnt=\count116
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/url/url.sty
|
||||
\Urlmuskip=\muskip11
|
||||
Package: url 2013/09/16 ver 3.4 Verb mode for urls, etc.
|
||||
)
|
||||
LaTeX Info: Redefining \url on input line 5041.
|
||||
\XeTeXLinkMargin=\dimen124
|
||||
\Fld@menulength=\count117
|
||||
\Field@Width=\dimen125
|
||||
\Fld@charsize=\dimen126
|
||||
Package hyperref Info: Hyper figures OFF on input line 6295.
|
||||
Package hyperref Info: Link nesting OFF on input line 6300.
|
||||
Package hyperref Info: Hyper index ON on input line 6303.
|
||||
Package hyperref Info: backreferencing OFF on input line 6310.
|
||||
Package hyperref Info: Link coloring OFF on input line 6315.
|
||||
Package hyperref Info: Link coloring with OCG OFF on input line 6320.
|
||||
Package hyperref Info: PDF/A mode OFF on input line 6325.
|
||||
LaTeX Info: Redefining \ref on input line 6365.
|
||||
LaTeX Info: Redefining \pageref on input line 6369.
|
||||
\Hy@abspage=\count118
|
||||
\c@Item=\count119
|
||||
\c@Hfootnote=\count120
|
||||
)
|
||||
|
||||
Package hyperref Message: Driver (autodetected): hpdftex.
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/hpdftex.def
|
||||
File: hpdftex.def 2012/11/06 v6.83m Hyperref driver for pdfTeX
|
||||
\Fld@listcount=\count121
|
||||
\c@bookmark@seq@number=\count122
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/oberdiek/rerunfilecheck.sty
|
||||
Package: rerunfilecheck 2011/04/15 v1.7 Rerun checks for auxiliary files (HO)
|
||||
Package uniquecounter Info: New unique counter `rerunfilecheck' on input line 2
|
||||
82.
|
||||
)
|
||||
\Hy@SectionHShift=\skip54
|
||||
)
|
||||
Package hyperref Info: Option `breaklinks' set `true' on input line 46.
|
||||
Package hyperref Info: Option `colorlinks' set `true' on input line 46.
|
||||
Package hyperref Info: Option `pdfmenubar' set `true' on input line 46.
|
||||
Package hyperref Info: Option `pdftoolbar' set `true' on input line 46.
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/fancyhdr/fancyhdr.sty
|
||||
\fancy@headwidth=\skip55
|
||||
\f@ncyO@elh=\skip56
|
||||
\f@ncyO@erh=\skip57
|
||||
\f@ncyO@olh=\skip58
|
||||
\f@ncyO@orh=\skip59
|
||||
\f@ncyO@elf=\skip60
|
||||
\f@ncyO@erf=\skip61
|
||||
\f@ncyO@olf=\skip62
|
||||
\f@ncyO@orf=\skip63
|
||||
)
|
||||
|
||||
Package Fancyhdr Warning: \fancyfoot's `E' option without twoside option is use
|
||||
less on input line 55.
|
||||
|
||||
\@indexfile=\write4
|
||||
\openout4 = `Project1.idx'.
|
||||
|
||||
Writing index file Project1.idx
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/idxlayout/idxlayout.sty
|
||||
Package: idxlayout 2012/03/30 v0.4d Configurable index layout
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/etoolbox/etoolbox.sty
|
||||
Package: etoolbox 2015/08/02 v2.2a e-TeX tools for LaTeX (JAW)
|
||||
\etb@tempcnta=\count123
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/tools/multicol.sty
|
||||
Package: multicol 2015/08/19 v1.8n multicolumn formatting (FMi)
|
||||
\c@tracingmulticols=\count124
|
||||
\mult@box=\box33
|
||||
\multicol@leftmargin=\dimen127
|
||||
\c@unbalance=\count125
|
||||
\c@collectmore=\count126
|
||||
\doublecol@number=\count127
|
||||
\multicoltolerance=\count128
|
||||
\multicolpretolerance=\count129
|
||||
\full@width=\dimen128
|
||||
\page@free=\dimen129
|
||||
\premulticols=\dimen130
|
||||
\postmulticols=\dimen131
|
||||
\multicolsep=\skip64
|
||||
\multicolbaselineskip=\skip65
|
||||
\partial@page=\box34
|
||||
\last@line=\box35
|
||||
\maxbalancingoverflow=\dimen132
|
||||
\mult@rightbox=\box36
|
||||
\mult@grightbox=\box37
|
||||
\mult@gfirstbox=\box38
|
||||
\mult@firstbox=\box39
|
||||
\@tempa=\box40
|
||||
\@tempa=\box41
|
||||
\@tempa=\box42
|
||||
\@tempa=\box43
|
||||
\@tempa=\box44
|
||||
\@tempa=\box45
|
||||
\@tempa=\box46
|
||||
\@tempa=\box47
|
||||
\@tempa=\box48
|
||||
\@tempa=\box49
|
||||
\@tempa=\box50
|
||||
\@tempa=\box51
|
||||
\@tempa=\box52
|
||||
\@tempa=\box53
|
||||
\@tempa=\box54
|
||||
\@tempa=\box55
|
||||
\@tempa=\box56
|
||||
\c@columnbadness=\count130
|
||||
\c@finalcolumnbadness=\count131
|
||||
\last@try=\dimen133
|
||||
\multicolovershoot=\dimen134
|
||||
\multicolundershoot=\dimen135
|
||||
\mult@nat@firstbox=\box57
|
||||
\colbreak@box=\box58
|
||||
\mc@col@check@num=\count132
|
||||
)
|
||||
\c@idxcols=\count133
|
||||
\indexcolsep=\skip66
|
||||
\indexrule=\skip67
|
||||
\ila@indentunit=\skip68
|
||||
\ila@initsep=\skip69
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/tocbibind/tocbibind.sty
|
||||
Package: tocbibind 2010/10/13 v1.5k extra ToC listings
|
||||
Package tocbibind Info: The document has section divisions on input line 50.
|
||||
|
||||
|
||||
Package tocbibind Note: Using section or other style headings.
|
||||
|
||||
) (/usr/share/texlive/texmf-dist/tex/latex/ms/ragged2e.sty
|
||||
Package: ragged2e 2009/05/21 v2.1 ragged2e Package (MS)
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/ms/everysel.sty
|
||||
Package: everysel 2011/10/28 v1.2 EverySelectfont Package (MS)
|
||||
)
|
||||
\CenteringLeftskip=\skip70
|
||||
\RaggedLeftLeftskip=\skip71
|
||||
\RaggedRightLeftskip=\skip72
|
||||
\CenteringRightskip=\skip73
|
||||
\RaggedLeftRightskip=\skip74
|
||||
\RaggedRightRightskip=\skip75
|
||||
\CenteringParfillskip=\skip76
|
||||
\RaggedLeftParfillskip=\skip77
|
||||
\RaggedRightParfillskip=\skip78
|
||||
\JustifyingParfillskip=\skip79
|
||||
\CenteringParindent=\skip80
|
||||
\RaggedLeftParindent=\skip81
|
||||
\RaggedRightParindent=\skip82
|
||||
\JustifyingParindent=\skip83
|
||||
)
|
||||
No file Project1.aux.
|
||||
\openout1 = `Project1.aux'.
|
||||
|
||||
LaTeX Font Info: Checking defaults for OML/cmm/m/it on input line 90.
|
||||
LaTeX Font Info: ... okay on input line 90.
|
||||
LaTeX Font Info: Checking defaults for T1/cmr/m/n on input line 90.
|
||||
LaTeX Font Info: ... okay on input line 90.
|
||||
LaTeX Font Info: Checking defaults for OT1/cmr/m/n on input line 90.
|
||||
LaTeX Font Info: ... okay on input line 90.
|
||||
LaTeX Font Info: Checking defaults for OMS/cmsy/m/n on input line 90.
|
||||
LaTeX Font Info: ... okay on input line 90.
|
||||
LaTeX Font Info: Checking defaults for OMX/cmex/m/n on input line 90.
|
||||
LaTeX Font Info: ... okay on input line 90.
|
||||
LaTeX Font Info: Checking defaults for U/cmr/m/n on input line 90.
|
||||
LaTeX Font Info: ... okay on input line 90.
|
||||
LaTeX Font Info: Checking defaults for PD1/pdf/m/n on input line 90.
|
||||
LaTeX Font Info: ... okay on input line 90.
|
||||
LaTeX Font Info: Try loading font information for T1+lmr on input line 90.
|
||||
(/usr/share/texmf/tex/latex/lm/t1lmr.fd
|
||||
File: t1lmr.fd 2009/10/30 v1.6 Font defs for Latin Modern
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/context/base/supp-pdf.mkii
|
||||
[Loading MPS to PDF converter (version 2006.09.02).]
|
||||
\scratchcounter=\count134
|
||||
\scratchdimen=\dimen136
|
||||
\scratchbox=\box59
|
||||
\nofMPsegments=\count135
|
||||
\nofMParguments=\count136
|
||||
\everyMPshowfont=\toks30
|
||||
\MPscratchCnt=\count137
|
||||
\MPscratchDim=\dimen137
|
||||
\MPnumerator=\count138
|
||||
\makeMPintoPDFobject=\count139
|
||||
\everyMPtoPDFconversion=\toks31
|
||||
)
|
||||
LaTeX Info: Redefining \microtypecontext on input line 90.
|
||||
Package microtype Info: Generating PDF output.
|
||||
Package microtype Info: Character protrusion enabled (level 2).
|
||||
Package microtype Info: Using default protrusion set `alltext'.
|
||||
Package microtype Info: Automatic font expansion enabled (level 2),
|
||||
(microtype) stretch: 20, shrink: 20, step: 1, non-selected.
|
||||
Package microtype Info: Using default expansion set `basictext'.
|
||||
Package microtype Info: No adjustment of tracking.
|
||||
Package microtype Info: No adjustment of interword spacing.
|
||||
Package microtype Info: No adjustment of character kerning.
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/microtype/mt-cmr.cfg
|
||||
File: mt-cmr.cfg 2013/05/19 v2.2 microtype config. file: Computer Modern Roman
|
||||
(RS)
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/oberdiek/epstopdf-base.sty
|
||||
Package: epstopdf-base 2010/02/09 v2.5 Base part for package epstopdf
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/oberdiek/grfext.sty
|
||||
Package: grfext 2010/08/19 v1.1 Manage graphics extensions (HO)
|
||||
)
|
||||
Package grfext Info: Graphics extension search list:
|
||||
(grfext) [.png,.pdf,.jpg,.mps,.jpeg,.jbig2,.jb2,.PNG,.PDF,.JPG,.JPE
|
||||
G,.JBIG2,.JB2,.eps]
|
||||
(grfext) \AppendGraphicsExtensions on input line 452.
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/latexconfig/epstopdf-sys.cfg
|
||||
File: epstopdf-sys.cfg 2010/07/13 v1.3 Configuration of (r)epstopdf for TeX Liv
|
||||
e
|
||||
))
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/ucs/ucsencs.def
|
||||
File: ucsencs.def 2011/01/21 Fixes to fontencodings LGR, T3
|
||||
)
|
||||
\AtBeginShipoutBox=\box60
|
||||
Package hyperref Info: Link coloring ON on input line 90.
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/hyperref/nameref.sty
|
||||
Package: nameref 2012/10/27 v2.43 Cross-referencing by name of section
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/generic/oberdiek/gettitlestring.sty
|
||||
Package: gettitlestring 2010/12/03 v1.4 Cleanup title references (HO)
|
||||
)
|
||||
\c@section@level=\count140
|
||||
)
|
||||
LaTeX Info: Redefining \ref on input line 90.
|
||||
LaTeX Info: Redefining \pageref on input line 90.
|
||||
LaTeX Info: Redefining \nameref on input line 90.
|
||||
\@outlinefile=\write5
|
||||
\openout5 = `Project1.out'.
|
||||
|
||||
|
||||
ABD: EverySelectfont initializing macros
|
||||
LaTeX Info: Redefining \selectfont on input line 90.
|
||||
LaTeX Font Info: Try loading font information for OT1+lmr on input line 155.
|
||||
|
||||
|
||||
(/usr/share/texmf/tex/latex/lm/ot1lmr.fd
|
||||
File: ot1lmr.fd 2009/10/30 v1.6 Font defs for Latin Modern
|
||||
)
|
||||
LaTeX Font Info: Try loading font information for OML+lmm on input line 155.
|
||||
|
||||
|
||||
(/usr/share/texmf/tex/latex/lm/omllmm.fd
|
||||
File: omllmm.fd 2009/10/30 v1.6 Font defs for Latin Modern
|
||||
)
|
||||
LaTeX Font Info: Try loading font information for OMS+lmsy on input line 155
|
||||
.
|
||||
|
||||
(/usr/share/texmf/tex/latex/lm/omslmsy.fd
|
||||
File: omslmsy.fd 2009/10/30 v1.6 Font defs for Latin Modern
|
||||
)
|
||||
LaTeX Font Info: Try loading font information for OMX+lmex on input line 155
|
||||
.
|
||||
|
||||
(/usr/share/texmf/tex/latex/lm/omxlmex.fd
|
||||
File: omxlmex.fd 2009/10/30 v1.6 Font defs for Latin Modern
|
||||
)
|
||||
LaTeX Font Info: External font `lmex10' loaded for size
|
||||
(Font) <10> on input line 155.
|
||||
LaTeX Font Info: External font `lmex10' loaded for size
|
||||
(Font) <7> on input line 155.
|
||||
LaTeX Font Info: External font `lmex10' loaded for size
|
||||
(Font) <5> on input line 155.
|
||||
LaTeX Font Info: Try loading font information for U+msa on input line 155.
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/amsfonts/umsa.fd
|
||||
File: umsa.fd 2013/01/14 v3.01 AMS symbols A
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/microtype/mt-msa.cfg
|
||||
File: mt-msa.cfg 2006/02/04 v1.1 microtype config. file: AMS symbols (a) (RS)
|
||||
)
|
||||
LaTeX Font Info: Try loading font information for U+msb on input line 155.
|
||||
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/amsfonts/umsb.fd
|
||||
File: umsb.fd 2013/01/14 v3.01 AMS symbols B
|
||||
)
|
||||
(/usr/share/texlive/texmf-dist/tex/latex/microtype/mt-msb.cfg
|
||||
File: mt-msb.cfg 2005/06/01 v1.0 microtype config. file: AMS symbols (b) (RS)
|
||||
)
|
||||
LaTeX Font Info: Try loading font information for OMS+lmr on input line 174.
|
||||
|
||||
|
||||
(/usr/share/texmf/tex/latex/lm/omslmr.fd
|
||||
File: omslmr.fd 2009/10/30 v1.6 Font defs for Latin Modern
|
||||
)
|
||||
LaTeX Font Info: Font shape `OMS/lmr/m/n' in size <8> not available
|
||||
(Font) Font shape `OMS/lmsy/m/n' tried instead on input line 174.
|
||||
[1
|
||||
|
||||
{/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map}]
|
||||
LaTeX Font Info: Try loading font information for T1+lmtt on input line 181.
|
||||
|
||||
(/usr/share/texmf/tex/latex/lm/t1lmtt.fd
|
||||
File: t1lmtt.fd 2009/10/30 v1.6 Font defs for Latin Modern
|
||||
)
|
||||
Package microtype Info: Loading generic settings for font family
|
||||
(microtype) `lmtt' (encoding: T1).
|
||||
(microtype) For optimal results, create family-specific settings.
|
||||
(microtype) See the microtype manual for details.
|
||||
[2] [3] [4]
|
||||
[5] (/usr/share/texlive/texmf-dist/tex/latex/ucs/data/uni-0.def
|
||||
File: uni-0.def 2013/05/13 UCS: Unicode data U+0000..U+00FF
|
||||
) [6]
|
||||
LaTeX Font Info: Font shape `OMS/lmr/m/n' in size <10> not available
|
||||
(Font) Font shape `OMS/lmsy/m/n' tried instead on input line 452.
|
||||
[7]
|
||||
[8]
|
||||
Package atveryend Info: Empty hook `BeforeClearDocument' on input line 551.
|
||||
[9]
|
||||
Package atveryend Info: Empty hook `AfterLastShipout' on input line 551.
|
||||
(./Project1.aux)
|
||||
Package atveryend Info: Executing hook `AtVeryEndDocument' on input line 551.
|
||||
|
||||
|
||||
*File List*
|
||||
article.cls 2014/09/29 v1.4h Standard LaTeX document class
|
||||
size10.clo 2014/09/29 v1.4h Standard LaTeX file (size option)
|
||||
relsize.sty 2013/03/29 ver 4.1
|
||||
makeidx.sty 2014/09/29 v1.0m Standard LaTeX package
|
||||
color.sty 1999/02/16
|
||||
color.cfg 2007/01/18 v1.5 color configuration of teTeX/TeXLive
|
||||
pdftex.def 2011/05/27 v0.06d Graphics/color for pdfTeX
|
||||
infwarerr.sty 2010/04/08 v1.3 Providing info/warning/error messages (HO)
|
||||
ltxcmds.sty 2011/11/09 v1.22 LaTeX kernel commands for general use (HO)
|
||||
setspace.sty 2011/12/19 v6.7a set line spacing
|
||||
amsmath.sty 2016/03/03 v2.15a AMS math features
|
||||
amstext.sty 2000/06/29 v2.01 AMS text
|
||||
amsgen.sty 1999/11/30 v2.0 generic functions
|
||||
amsbsy.sty 1999/11/29 v1.2d Bold Symbols
|
||||
amsopn.sty 1999/12/14 v2.01 operator names
|
||||
amsfonts.sty 2013/01/14 v3.01 Basic AMSFonts support
|
||||
amssymb.sty 2013/01/14 v3.01 AMS font symbols
|
||||
xcolor.sty 2007/01/21 v2.11 LaTeX color extensions (UK)
|
||||
color.cfg 2007/01/18 v1.5 color configuration of teTeX/TeXLive
|
||||
colortbl.sty 2012/02/13 v1.0a Color table columns (DPC)
|
||||
array.sty 2014/10/28 v2.4c Tabular extension package (FMi)
|
||||
bm.sty 2014/10/28 v1.1c Bold Symbol Support (DPC/FMi)
|
||||
ltablex.sty 2014/08/13 v1.1 Modified tabularx
|
||||
longtable.sty 2014/10/28 v4.11 Multi-page Table package (DPC)
|
||||
tabularx.sty 2014/10/28 v2.10 `tabularx' package (DPC)
|
||||
microtype.sty 2013/05/23 v2.5a Micro-typographical refinements (RS)
|
||||
keyval.sty 2014/10/28 v1.15 key=value parser (DPC)
|
||||
microtype-pdftex.def 2013/05/23 v2.5a Definitions specific to pdftex (RS)
|
||||
microtype.cfg 2013/05/23 v2.5a microtype main configuration file (RS)
|
||||
graphicx.sty 2014/10/28 v1.0g Enhanced LaTeX Graphics (DPC,SPQR)
|
||||
graphics.sty 2016/01/03 v1.0q Standard LaTeX Graphics (DPC,SPQR)
|
||||
trig.sty 2016/01/03 v1.10 sin cos tan (DPC)
|
||||
graphics.cfg 2010/04/23 v1.9 graphics configuration of TeX Live
|
||||
fancyvrb.sty 2008/02/07
|
||||
fontenc.sty
|
||||
t1enc.def 2005/09/27 v1.99g Standard LaTeX file
|
||||
ucs.sty 2013/05/11 v2.2 UCS: Unicode input support
|
||||
uni-global.def 2013/05/13 UCS: Unicode global data
|
||||
inputenc.sty 2015/03/17 v1.2c Input encoding file
|
||||
utf8x.def 2004/10/17 UCS: Input encoding UTF-8
|
||||
lmodern.sty 2009/10/30 v1.6 Latin Modern Fonts
|
||||
hyperref.sty 2012/11/06 v6.83m Hypertext links for LaTeX
|
||||
hobsub-hyperref.sty 2012/05/28 v1.13 Bundle oberdiek, subset hyperref (HO)
|
||||
hobsub-generic.sty 2012/05/28 v1.13 Bundle oberdiek, subset generic (HO)
|
||||
hobsub.sty 2012/05/28 v1.13 Construct package bundles (HO)
|
||||
ifluatex.sty 2010/03/01 v1.3 Provides the ifluatex switch (HO)
|
||||
ifvtex.sty 2010/03/01 v1.5 Detect VTeX and its facilities (HO)
|
||||
intcalc.sty 2007/09/27 v1.1 Expandable calculations with integers (HO)
|
||||
ifpdf.sty 2011/01/30 v2.3 Provides the ifpdf switch (HO)
|
||||
etexcmds.sty 2011/02/16 v1.5 Avoid name clashes with e-TeX commands (HO)
|
||||
kvsetkeys.sty 2012/04/25 v1.16 Key value parser (HO)
|
||||
kvdefinekeys.sty 2011/04/07 v1.3 Define keys (HO)
|
||||
pdftexcmds.sty 2011/11/29 v0.20 Utility functions of pdfTeX for LuaTeX (HO)
|
||||
pdfescape.sty 2011/11/25 v1.13 Implements pdfTeX's escape features (HO)
|
||||
bigintcalc.sty 2012/04/08 v1.3 Expandable calculations on big integers (HO)
|
||||
bitset.sty 2011/01/30 v1.1 Handle bit-vector datatype (HO)
|
||||
uniquecounter.sty 2011/01/30 v1.2 Provide unlimited unique counter (HO)
|
||||
letltxmacro.sty 2010/09/02 v1.4 Let assignment for LaTeX macros (HO)
|
||||
hopatch.sty 2012/05/28 v1.2 Wrapper for package hooks (HO)
|
||||
xcolor-patch.sty 2011/01/30 xcolor patch
|
||||
atveryend.sty 2011/06/30 v1.8 Hooks at the very end of document (HO)
|
||||
atbegshi.sty 2011/10/05 v1.16 At begin shipout hook (HO)
|
||||
refcount.sty 2011/10/16 v3.4 Data extraction from label references (HO)
|
||||
hycolor.sty 2011/01/30 v1.7 Color options for hyperref/bookmark (HO)
|
||||
ifxetex.sty 2010/09/12 v0.6 Provides ifxetex conditional
|
||||
auxhook.sty 2011/03/04 v1.3 Hooks for auxiliary files (HO)
|
||||
kvoptions.sty 2011/06/30 v3.11 Key value format for package options (HO)
|
||||
pd1enc.def 2012/11/06 v6.83m Hyperref: PDFDocEncoding definition (HO)
|
||||
hyperref.cfg 2002/06/06 v1.2 hyperref configuration of TeXLive
|
||||
url.sty 2013/09/16 ver 3.4 Verb mode for urls, etc.
|
||||
hpdftex.def 2012/11/06 v6.83m Hyperref driver for pdfTeX
|
||||
rerunfilecheck.sty 2011/04/15 v1.7 Rerun checks for auxiliary files (HO)
|
||||
fancyhdr.sty
|
||||
idxlayout.sty 2012/03/30 v0.4d Configurable index layout
|
||||
etoolbox.sty 2015/08/02 v2.2a e-TeX tools for LaTeX (JAW)
|
||||
multicol.sty 2015/08/19 v1.8n multicolumn formatting (FMi)
|
||||
tocbibind.sty 2010/10/13 v1.5k extra ToC listings
|
||||
ragged2e.sty 2009/05/21 v2.1 ragged2e Package (MS)
|
||||
everysel.sty 2011/10/28 v1.2 EverySelectfont Package (MS)
|
||||
t1lmr.fd 2009/10/30 v1.6 Font defs for Latin Modern
|
||||
supp-pdf.mkii
|
||||
mt-cmr.cfg 2013/05/19 v2.2 microtype config. file: Computer Modern Roman (
|
||||
RS)
|
||||
epstopdf-base.sty 2010/02/09 v2.5 Base part for package epstopdf
|
||||
grfext.sty 2010/08/19 v1.1 Manage graphics extensions (HO)
|
||||
epstopdf-sys.cfg 2010/07/13 v1.3 Configuration of (r)epstopdf for TeX Live
|
||||
ucsencs.def 2011/01/21 Fixes to fontencodings LGR, T3
|
||||
nameref.sty 2012/10/27 v2.43 Cross-referencing by name of section
|
||||
gettitlestring.sty 2010/12/03 v1.4 Cleanup title references (HO)
|
||||
ot1lmr.fd 2009/10/30 v1.6 Font defs for Latin Modern
|
||||
omllmm.fd 2009/10/30 v1.6 Font defs for Latin Modern
|
||||
omslmsy.fd 2009/10/30 v1.6 Font defs for Latin Modern
|
||||
omxlmex.fd 2009/10/30 v1.6 Font defs for Latin Modern
|
||||
umsa.fd 2013/01/14 v3.01 AMS symbols A
|
||||
mt-msa.cfg 2006/02/04 v1.1 microtype config. file: AMS symbols (a) (RS)
|
||||
umsb.fd 2013/01/14 v3.01 AMS symbols B
|
||||
mt-msb.cfg 2005/06/01 v1.0 microtype config. file: AMS symbols (b) (RS)
|
||||
omslmr.fd 2009/10/30 v1.6 Font defs for Latin Modern
|
||||
t1lmtt.fd 2009/10/30 v1.6 Font defs for Latin Modern
|
||||
uni-0.def 2013/05/13 UCS: Unicode data U+0000..U+00FF
|
||||
***********
|
||||
|
||||
Package atveryend Info: Executing hook `AtEndAfterFileList' on input line 551.
|
||||
|
||||
Package rerunfilecheck Warning: File `Project1.out' has changed.
|
||||
(rerunfilecheck) Rerun to get outlines right
|
||||
(rerunfilecheck) or use package `bookmark'.
|
||||
|
||||
Package rerunfilecheck Info: Checksums for `Project1.out':
|
||||
(rerunfilecheck) Before: <no file>
|
||||
(rerunfilecheck) After: D41D8CD98F00B204E9800998ECF8427E;0.
|
||||
Package atveryend Info: Empty hook `AtVeryVeryEnd' on input line 551.
|
||||
)
|
||||
Here is how much of TeX's memory you used:
|
||||
10632 strings out of 493029
|
||||
155761 string characters out of 6136233
|
||||
291311 words of memory out of 5000000
|
||||
13835 multiletter control sequences out of 15000+600000
|
||||
66983 words of font info for 101 fonts, out of 8000000 for 9000
|
||||
1141 hyphenation exceptions out of 8191
|
||||
41i,15n,43p,1001b,479s stack positions out of 5000i,500n,10000p,200000b,80000s
|
||||
{/usr/share/texmf/fonts/enc/dvips/lm/lm-rm.enc}{/usr/share/texmf/fonts/enc/dv
|
||||
ips/lm/lm-mathsy.enc}{/usr/share/texmf/fonts/enc/dvips/lm/lm-mathit.enc}{/usr/s
|
||||
hare/texmf/fonts/enc/dvips/lm/lm-ec.enc}{/usr/share/texmf/fonts/enc/dvips/lm/lm
|
||||
-mathex.enc}</usr/share/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/cmbx1
|
||||
0.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/cmmib10.pfb
|
||||
></usr/share/texmf/fonts/type1/public/lm/lmbx10.pfb></usr/share/texmf/fonts/typ
|
||||
e1/public/lm/lmbx12.pfb></usr/share/texmf/fonts/type1/public/lm/lmex10.pfb></us
|
||||
r/share/texmf/fonts/type1/public/lm/lmmi10.pfb></usr/share/texmf/fonts/type1/pu
|
||||
blic/lm/lmmi7.pfb></usr/share/texmf/fonts/type1/public/lm/lmr10.pfb></usr/share
|
||||
/texmf/fonts/type1/public/lm/lmr7.pfb></usr/share/texmf/fonts/type1/public/lm/l
|
||||
mr8.pfb></usr/share/texmf/fonts/type1/public/lm/lmr9.pfb></usr/share/texmf/font
|
||||
s/type1/public/lm/lmri10.pfb></usr/share/texmf/fonts/type1/public/lm/lmsy10.pfb
|
||||
></usr/share/texmf/fonts/type1/public/lm/lmsy7.pfb></usr/share/texmf/fonts/type
|
||||
1/public/lm/lmsy8.pfb></usr/share/texmf/fonts/type1/public/lm/lmtt10.pfb></usr/
|
||||
share/texlive/texmf-dist/fonts/type1/public/amsfonts/symbols/msbm10.pfb>
|
||||
Output written on Project1.pdf (9 pages, 281972 bytes).
|
||||
PDF statistics:
|
||||
168 PDF objects out of 1000 (max. 8388607)
|
||||
138 compressed objects within 2 object streams
|
||||
29 named destinations out of 1000 (max. 500000)
|
||||
19969 words of extra memory for PDF output out of 20736 (max. 10000000)
|
||||
|
||||
@@ -155,7 +155,7 @@ Project 1 on Machine Learning, deadline October 5, 2020
|
||||
|
||||
% --- begin date ---
|
||||
\begin{center}
|
||||
Sep 7, 2020
|
||||
Sep 16, 2020
|
||||
\end{center}
|
||||
% --- end date ---
|
||||
|
||||
@@ -370,19 +370,15 @@ Note also that when you calculate the bias, in all applications you don't know t
|
||||
The aim here is to write your own code for another widely popular
|
||||
resampling technique, the so-called cross-validation method. Again,
|
||||
before you start with cross-validation approach, you should scale your
|
||||
data and split it in test and training data as you did earlier.
|
||||
Perform a resampling of the data where you split the data in training
|
||||
data and test data using for example
|
||||
|
||||
|
||||
data.
|
||||
|
||||
Implement the $k$-fold cross-validation algorithm (write your own
|
||||
code) and evaluate again the MSE function resulting
|
||||
from the test data. You can compare your own code with that from
|
||||
from the test folds. You can compare your own code with that from
|
||||
\textbf{Scikit-Learn} if needed.
|
||||
|
||||
Compare the MSE you get from your cross-validation code with the one you got from your \textbf{bootstrap} code.
|
||||
You can also compare your own cross-validation code with the one provided by \textbf{Scikit-Learn}.
|
||||
Compare the MSE you get from your cross-validation code with the one you got from your \textbf{bootstrap} code. Comment your results. Try $5-10$ folds.
|
||||
You can also compare your own cross-validation code with the one provided by \textbf{Scikit-Learn}.
|
||||
|
||||
|
||||
\paragraph{Part d): Ridge Regression on the Franke function with resampling.}
|
||||
|
||||
Binary file not shown.
@@ -125,7 +125,7 @@ Project 1 on Machine Learning, deadline October 5, 2020
|
||||
|
||||
% --- begin date ---
|
||||
\begin{center}
|
||||
Sep 7, 2020
|
||||
Sep 16, 2020
|
||||
\end{center}
|
||||
% --- end date ---
|
||||
|
||||
@@ -227,7 +227,7 @@ We will generate our own dataset for a function
|
||||
$\mathrm{FrankeFunction}(x,y)$ with $x,y \in [0,1]$. The function
|
||||
$f(x,y)$ is the Franke function. You should explore also the addition
|
||||
of an added stochastic noise to this function using the normal
|
||||
distribution $\cal{N}(0,1)$.
|
||||
distribution $N(0,1)$.
|
||||
|
||||
\emph{Write your own code} (using either a matrix inversion or a singular
|
||||
value decomposition from e.g., \textbf{numpy} ) or use your code from
|
||||
@@ -340,19 +340,15 @@ Note also that when you calculate the bias, in all applications you don't know t
|
||||
The aim here is to write your own code for another widely popular
|
||||
resampling technique, the so-called cross-validation method. Again,
|
||||
before you start with cross-validation approach, you should scale your
|
||||
data and split it in test and training data as you did earlier.
|
||||
Perform a resampling of the data where you split the data in training
|
||||
data and test data using for example
|
||||
|
||||
|
||||
data.
|
||||
|
||||
Implement the $k$-fold cross-validation algorithm (write your own
|
||||
code) and evaluate again the MSE function resulting
|
||||
from the test data. You can compare your own code with that from
|
||||
from the test folds. You can compare your own code with that from
|
||||
\textbf{Scikit-Learn} if needed.
|
||||
|
||||
Compare the MSE you get from your cross-validation code with the one you got from your \textbf{bootstrap} code.
|
||||
You can also compare your own cross-validation code with the one provided by \textbf{Scikit-Learn}.
|
||||
Compare the MSE you get from your cross-validation code with the one you got from your \textbf{bootstrap} code. Comment your results. Try $5-10$ folds.
|
||||
You can also compare your own cross-validation code with the one provided by \textbf{Scikit-Learn}.
|
||||
|
||||
|
||||
\paragraph{Part d): Ridge Regression on the Franke function with resampling.}
|
||||
|
||||
@@ -0,0 +1,740 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<!-- dom:TITLE: Homework 2, weeks 36 and 37 -->\n",
|
||||
"# Homework 2, weeks 36 and 37\n",
|
||||
"<!-- dom:AUTHOR: [Data Analysis and Machine Learning FYS-STK3155/FYS4155](http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html) at Department of Physics, University of Oslo, Norway -->\n",
|
||||
"<!-- Author: --> \n",
|
||||
"**[Data Analysis and Machine Learning FYS-STK3155/FYS4155](http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html)**, Department of Physics, University of Oslo, Norway\n",
|
||||
"\n",
|
||||
"Date: **Sep 8, 2020**\n",
|
||||
"\n",
|
||||
"Copyright 1999-2020, [Data Analysis and Machine Learning FYS-STK3155/FYS4155](http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html). Released under CC Attribution-NonCommercial 4.0 license\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"<!-- --- begin exercise --- -->\n",
|
||||
"\n",
|
||||
"## Exercise 1: Adding Ridge and Lasso Regression\n",
|
||||
"\n",
|
||||
"This exercise is a continuation of exercise 3 from exercise set 1 (week 35). We will\n",
|
||||
"use the same function to generate our data set, still staying with a\n",
|
||||
"simple function $y(x)$ which we want to fit using linear regression,\n",
|
||||
"but now extending the analysis to include the Ridge and the Lasso\n",
|
||||
"regression methods. You can use the code under the Regression as an example on how to use the Ridge and the Lasso methods, see the [regression slides](https://compphysics.github.io/MachineLearning/doc/pub/Regression/html/Regression-bs.html)). \n",
|
||||
"\n",
|
||||
"We will thus again generate our own dataset for a function $y(x)$ where \n",
|
||||
"$x \\in [0,1]$ and defined by random numbers computed with the uniform\n",
|
||||
"distribution. The function $y$ is a quadratic polynomial in $x$ with\n",
|
||||
"added stochastic noise according to the normal distribution $\\cal{N}(0,1)$.\n",
|
||||
"\n",
|
||||
"The following simple Python instructions define our $x$ and $y$ values (with 100 data points)."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"x = np.random.rand(100)\n",
|
||||
"y = 2.0+5*x*x+0.1*np.random.randn(100)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"**a)**\n",
|
||||
"Write your own code for the Ridge method (see chapter 3.4 of Hastie *et al.*, equations (3.43) and (3.44)) and compute the parametrization for different values of $\\lambda$. Compare and analyze your results with those from exercise 3. Study the dependence on $\\lambda$ while also varying the strength of the noise in your expression for $y(x)$.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"<!-- --- begin solution of exercise --- -->\n",
|
||||
"**Solution.**\n",
|
||||
"The code here allows you to perform your own Ridge calculation and perform calculations for various values of the regularization parameter $\\lambda$. This program can easily be extended upon."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 2,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"%matplotlib inline\n",
|
||||
"\n",
|
||||
"import os\n",
|
||||
"import numpy as np\n",
|
||||
"import pandas as pd\n",
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"from sklearn.model_selection import train_test_split\n",
|
||||
"from sklearn.preprocessing import StandardScaler\n",
|
||||
"\n",
|
||||
"def R2(y_data, y_model):\n",
|
||||
" return 1 - np.sum((y_data - y_model) ** 2) / np.sum((y_data - np.mean(y_data)) ** 2)\n",
|
||||
"def MSE(y_data,y_model):\n",
|
||||
" n = np.size(y_model)\n",
|
||||
" return np.sum((y_data-y_model)**2)/n\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# A seed just to ensure that the random numbers are the same for every run.\n",
|
||||
"# Useful for eventual debugging.\n",
|
||||
"np.random.seed(3155)\n",
|
||||
"\n",
|
||||
"x = np.random.rand(100)\n",
|
||||
"y = 2.0+5*x*x+0.1*np.random.randn(100)\n",
|
||||
"\n",
|
||||
"# number of features p (here degree of polynomial\n",
|
||||
"p = 3\n",
|
||||
"# The design matrix now as function of a given polynomial\n",
|
||||
"X = np.zeros((len(x),p))\n",
|
||||
"X[:,0] = 1.0\n",
|
||||
"X[:,1] = x\n",
|
||||
"X[:,2] = x*x\n",
|
||||
"# We split the data in test and training data\n",
|
||||
"X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)\n",
|
||||
"scaler = StandardScaler()\n",
|
||||
"scaler.fit(X_train)\n",
|
||||
"X_train_scaled = scaler.transform(X_train)\n",
|
||||
"X_test_scaled = scaler.transform(X_test)\n",
|
||||
"\n",
|
||||
"# matrix inversion to find beta\n",
|
||||
"OLSbeta = np.linalg.inv(X_train.T @ X_train) @ X_train.T @ y_train\n",
|
||||
"print(OLSbeta)\n",
|
||||
"# and then make the prediction\n",
|
||||
"ytildeOLS = X_train @ OLSbeta\n",
|
||||
"print(\"Training R2 for OLS\")\n",
|
||||
"print(R2(y_train,ytildeOLS))\n",
|
||||
"print(\"Training MSE for OLS\")\n",
|
||||
"print(MSE(y_train,ytildeOLS))\n",
|
||||
"ypredictOLS = X_test @ OLSbeta\n",
|
||||
"print(\"Test R2 for OLS\")\n",
|
||||
"print(R2(y_test,ypredictOLS))\n",
|
||||
"print(\"Test MSE OLS\")\n",
|
||||
"print(MSE(y_test,ypredictOLS))\n",
|
||||
"\n",
|
||||
"# Repeat now for Ridge regression and various values of the regularization parameter\n",
|
||||
"I = np.eye(p,p)\n",
|
||||
"# Decide which values of lambda to use\n",
|
||||
"nlambdas = 20\n",
|
||||
"MSEPredict = np.zeros(nlambdas)\n",
|
||||
"MSETrain = np.zeros(nlambdas)\n",
|
||||
"lambdas = np.logspace(-4, 1, nlambdas)\n",
|
||||
"for i in range(nlambdas):\n",
|
||||
" lmb = lambdas[i]\n",
|
||||
" Ridgebeta = np.linalg.inv(X_train.T @ X_train+lmb*I) @ X_train.T @ y_train\n",
|
||||
" # and then make the prediction\n",
|
||||
" ytildeRidge = X_train @ Ridgebeta\n",
|
||||
" ypredictRidge = X_test @ Ridgebeta\n",
|
||||
" MSEPredict[i] = MSE(y_test,ypredictRidge)\n",
|
||||
" MSETrain[i] = MSE(y_train,ytildeRidge)\n",
|
||||
"# Now plot the results\n",
|
||||
"plt.figure()\n",
|
||||
"plt.plot(np.log10(lambdas), MSETrain, label = 'MSE Ridge train')\n",
|
||||
"plt.plot(np.log10(lambdas), MSEPredict, 'r--', label = 'MSE Ridge Test')\n",
|
||||
"plt.xlabel('log10(lambda)')\n",
|
||||
"plt.ylabel('MSE')\n",
|
||||
"plt.legend()\n",
|
||||
"plt.show()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<!-- --- end solution of exercise --- -->\n",
|
||||
"\n",
|
||||
"**b)**\n",
|
||||
"Repeat the above but using the functionality of **Scikit-Learn**. Compare your code with the results from **Scikit-Learn**. Remember to run with the same random numbers for generating $x$ and $y$.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"<!-- --- begin solution of exercise --- -->\n",
|
||||
"**Solution.**\n",
|
||||
"To use **scikit-learn** with Ridge, we simply need to add the relevant function **Ridge()**, as done in the code here."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 3,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import numpy as np\n",
|
||||
"import pandas as pd\n",
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"from sklearn.model_selection import train_test_split\n",
|
||||
"from sklearn.preprocessing import StandardScaler\n",
|
||||
"import sklearn.linear_model as skl\n",
|
||||
"\n",
|
||||
"def R2(y_data, y_model):\n",
|
||||
" return 1 - np.sum((y_data - y_model) ** 2) / np.sum((y_data - np.mean(y_data)) ** 2)\n",
|
||||
"def MSE(y_data,y_model):\n",
|
||||
" n = np.size(y_model)\n",
|
||||
" return np.sum((y_data-y_model)**2)/n\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# A seed just to ensure that the random numbers are the same for every run.\n",
|
||||
"# Useful for eventual debugging.\n",
|
||||
"np.random.seed(3155)\n",
|
||||
"\n",
|
||||
"x = np.random.rand(100)\n",
|
||||
"y = 2.0+5*x*x+0.1*np.random.randn(100)\n",
|
||||
"\n",
|
||||
"# number of features p (here degree of polynomial\n",
|
||||
"p = 3\n",
|
||||
"# The design matrix now as function of a given polynomial\n",
|
||||
"X = np.zeros((len(x),p))\n",
|
||||
"X[:,0] = 1.0\n",
|
||||
"X[:,1] = x\n",
|
||||
"X[:,2] = x*x\n",
|
||||
"# We split the data in test and training data\n",
|
||||
"X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)\n",
|
||||
"scaler = StandardScaler()\n",
|
||||
"scaler.fit(X_train)\n",
|
||||
"X_train_scaled = scaler.transform(X_train)\n",
|
||||
"X_test_scaled = scaler.transform(X_test)\n",
|
||||
"\n",
|
||||
"# matrix inversion to find beta\n",
|
||||
"OLSbeta = np.linalg.inv(X_train.T @ X_train) @ X_train.T @ y_train\n",
|
||||
"print(OLSbeta)\n",
|
||||
"# and then make the prediction\n",
|
||||
"ytildeOLS = X_train @ OLSbeta\n",
|
||||
"print(\"Training R2 for OLS\")\n",
|
||||
"print(R2(y_train,ytildeOLS))\n",
|
||||
"print(\"Training MSE for OLS\")\n",
|
||||
"print(MSE(y_train,ytildeOLS))\n",
|
||||
"ypredictOLS = X_test @ OLSbeta\n",
|
||||
"print(\"Test R2 for OLS\")\n",
|
||||
"print(R2(y_test,ypredictOLS))\n",
|
||||
"print(\"Test MSE OLS\")\n",
|
||||
"print(MSE(y_test,ypredictOLS))\n",
|
||||
"\n",
|
||||
"# Repeat now for Ridge regression and various values of the regularization parameter\n",
|
||||
"I = np.eye(p,p)\n",
|
||||
"# Decide which values of lambda to use\n",
|
||||
"nlambdas = 100\n",
|
||||
"MSEPredict = np.zeros(nlambdas)\n",
|
||||
"MSEPredictSKL = np.zeros(nlambdas)\n",
|
||||
"MSETrain = np.zeros(nlambdas)\n",
|
||||
"lambdas = np.logspace(-4, 0, nlambdas)\n",
|
||||
"for i in range(nlambdas):\n",
|
||||
" lmb = lambdas[i]\n",
|
||||
" # add ridge\n",
|
||||
" clf_ridge = skl.Ridge(alpha=lmb).fit(X_train, y_train)\n",
|
||||
" yridge = clf_ridge.predict(X_test)\n",
|
||||
" Ridgebeta = np.linalg.inv(X_train.T @ X_train+lmb*I) @ X_train.T @ y_train\n",
|
||||
" # and then make the prediction\n",
|
||||
" ytildeRidge = X_train @ Ridgebeta\n",
|
||||
" ypredictRidge = X_test @ Ridgebeta\n",
|
||||
" MSEPredict[i] = MSE(y_test,ypredictRidge)\n",
|
||||
" MSEPredictSKL[i] = MSE(y_test,yridge)\n",
|
||||
" MSETrain[i] = MSE(y_train,ytildeRidge)\n",
|
||||
"#then plot the results\n",
|
||||
"plt.figure()\n",
|
||||
"plt.plot(np.log10(lambdas), MSETrain, label = 'MSE Ridge train')\n",
|
||||
"plt.plot(np.log10(lambdas), MSEPredict, 'r--', label = 'MSE Ridge Test')\n",
|
||||
"plt.plot(np.log10(lambdas), MSEPredictSKL, 'g--', label = 'MSE Ridge sickit-learn Test')\n",
|
||||
"plt.xlabel('log10(lambda)')\n",
|
||||
"plt.ylabel('MSE')\n",
|
||||
"plt.legend()\n",
|
||||
"plt.show()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<!-- --- end solution of exercise --- -->\n",
|
||||
"\n",
|
||||
"**c)**\n",
|
||||
"Our next step is to study the variance of the parameters $\\beta_1$ and $\\beta_2$ (assuming that we are parameterizing our function with a second-order polynomial). We will use standard linear regression and the Ridge regression. You can now opt for either writing your own function or using **Scikit-Learn** to find the parameters $\\beta$. From your results calculate the variance of these parameters (recall that this is equal to the diagonal elements of the matrix $(\\hat{X}^T\\hat{X})+\\lambda\\hat{I})^{-1}$). Discuss the results of these variances as functions of $\\lambda$. In particular, try to link your discussion with the discussion in Hastie *et al.* and their figures 3.10 and 3.11. **Scikit-Learn** may not provide the variance of the parameters $\\beta$. This needs to be checked. With your own code you can however do so.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"<!-- --- begin solution of exercise --- -->\n",
|
||||
"**Solution.**"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 4,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import numpy as np\n",
|
||||
"import pandas as pd\n",
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"from sklearn.model_selection import train_test_split\n",
|
||||
"from sklearn.preprocessing import StandardScaler\n",
|
||||
"import sklearn.linear_model as skl\n",
|
||||
"\n",
|
||||
"def R2(y_data, y_model):\n",
|
||||
" return 1 - np.sum((y_data - y_model) ** 2) / np.sum((y_data - np.mean(y_data)) ** 2)\n",
|
||||
"def MSE(y_data,y_model):\n",
|
||||
" n = np.size(y_model)\n",
|
||||
" return np.sum((y_data-y_model)**2)/n\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# A seed just to ensure that the random numbers are the same for every run.\n",
|
||||
"# Useful for eventual debugging.\n",
|
||||
"np.random.seed(3155)\n",
|
||||
"\n",
|
||||
"x = np.random.rand(100)\n",
|
||||
"y = 2.0+5*x*x+0.1*np.random.randn(100)\n",
|
||||
"\n",
|
||||
"# number of features p (here degree of polynomial\n",
|
||||
"p = 3\n",
|
||||
"# The design matrix now as function of a given polynomial\n",
|
||||
"X = np.zeros((len(x),p))\n",
|
||||
"X[:,0] = 1.0\n",
|
||||
"X[:,1] = x\n",
|
||||
"X[:,2] = x*x\n",
|
||||
"# We split the data in test and training data\n",
|
||||
"X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)\n",
|
||||
"scaler = StandardScaler()\n",
|
||||
"scaler.fit(X_train)\n",
|
||||
"X_train_scaled = scaler.transform(X_train)\n",
|
||||
"X_test_scaled = scaler.transform(X_test)\n",
|
||||
"\n",
|
||||
"# matrix inversion to find beta\n",
|
||||
"OLSbeta = np.linalg.inv(X_train.T @ X_train) @ X_train.T @ y_train\n",
|
||||
"print(OLSbeta)\n",
|
||||
"# The variance is given by the inverse of the matrix X^TX\n",
|
||||
"print(np.linalg.inv(X_train.T @ X_train))\n",
|
||||
"\n",
|
||||
"# Repeat now for Ridge regression and various values of the regularization parameter\n",
|
||||
"I = np.eye(p,p)\n",
|
||||
"# Decide which values of lambda to use\n",
|
||||
"nlambdas = 10\n",
|
||||
"MSEPredict = np.zeros(nlambdas)\n",
|
||||
"MSEPredictSKL = np.zeros(nlambdas)\n",
|
||||
"MSETrain = np.zeros(nlambdas)\n",
|
||||
"lambdas = np.logspace(-4, 0, nlambdas)\n",
|
||||
"for i in range(nlambdas):\n",
|
||||
" lmb = lambdas[i]\n",
|
||||
" Ridgebeta = np.linalg.inv(X_train.T @ X_train+lmb*I) @ X_train.T @ y_train\n",
|
||||
" print(np.linalg.inv(X_train.T @ X_train+lmb*I))"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<!-- --- end solution of exercise --- -->\n",
|
||||
"\n",
|
||||
"**d)**\n",
|
||||
"Repeat the previous step but add now the Lasso method, see equation (3.53) of Hastie *et al.*. Discuss your results and compare with standard regression and the Ridge regression results. You can write your own code or use the functionality of **scikit-learn**. We recommend the latter since we have not yet discussed how to solve the Lasso equations numerically. Also, you do not need to compute the variance of the parameters $\\beta$ but you can extract their values and study their behavior as functions of the regularization parameter $\\lambda$.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"<!-- --- begin solution of exercise --- -->\n",
|
||||
"**Solution.**"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 5,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import numpy as np\n",
|
||||
"import pandas as pd\n",
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"from sklearn.model_selection import train_test_split\n",
|
||||
"from sklearn.preprocessing import StandardScaler\n",
|
||||
"import sklearn.linear_model as skl\n",
|
||||
"#from sklearn.linear_model import LinearRegression, Ridge, Lasso\n",
|
||||
"def R2(y_data, y_model):\n",
|
||||
" return 1 - np.sum((y_data - y_model) ** 2) / np.sum((y_data - np.mean(y_data)) ** 2)\n",
|
||||
"def MSE(y_data,y_model):\n",
|
||||
" n = np.size(y_model)\n",
|
||||
" return np.sum((y_data-y_model)**2)/n\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"# A seed just to ensure that the random numbers are the same for every run.\n",
|
||||
"# Useful for eventual debugging.\n",
|
||||
"np.random.seed(3155)\n",
|
||||
"\n",
|
||||
"x = np.random.rand(100)\n",
|
||||
"y = 2.0+5*x*x+0.1*np.random.randn(100)\n",
|
||||
"\n",
|
||||
"# number of features p (here degree of polynomial\n",
|
||||
"p = 3\n",
|
||||
"# The design matrix now as function of a given polynomial\n",
|
||||
"X = np.zeros((len(x),p))\n",
|
||||
"X[:,0] = 1.0\n",
|
||||
"X[:,1] = x\n",
|
||||
"X[:,2] = x*x\n",
|
||||
"# We split the data in test and training data\n",
|
||||
"X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)\n",
|
||||
"scaler = StandardScaler()\n",
|
||||
"scaler.fit(X_train)\n",
|
||||
"X_train_scaled = scaler.transform(X_train)\n",
|
||||
"X_test_scaled = scaler.transform(X_test)\n",
|
||||
"\n",
|
||||
"# matrix inversion to find beta\n",
|
||||
"OLSbeta = np.linalg.inv(X_train.T @ X_train) @ X_train.T @ y_train\n",
|
||||
"print(OLSbeta)\n",
|
||||
"# and then make the prediction\n",
|
||||
"ytildeOLS = X_train @ OLSbeta\n",
|
||||
"print(\"Training R2 for OLS\")\n",
|
||||
"print(R2(y_train,ytildeOLS))\n",
|
||||
"print(\"Training MSE for OLS\")\n",
|
||||
"print(MSE(y_train,ytildeOLS))\n",
|
||||
"ypredictOLS = X_test @ OLSbeta\n",
|
||||
"print(\"Test R2 for OLS\")\n",
|
||||
"print(R2(y_test,ypredictOLS))\n",
|
||||
"print(\"Test MSE OLS\")\n",
|
||||
"print(MSE(y_test,ypredictOLS))\n",
|
||||
"\n",
|
||||
"# Repeat now for Ridge regression and various values of the regularization parameter\n",
|
||||
"I = np.eye(p,p)\n",
|
||||
"# Decide which values of lambda to use\n",
|
||||
"nlambdas = 100\n",
|
||||
"MSEPredictLasso = np.zeros(nlambdas)\n",
|
||||
"MSEPredictRidge = np.zeros(nlambdas)\n",
|
||||
"lambdas = np.logspace(-4, 0, nlambdas)\n",
|
||||
"for i in range(nlambdas):\n",
|
||||
" lmb = lambdas[i]\n",
|
||||
" # add ridge\n",
|
||||
" clf_ridge = skl.Ridge(alpha=lmb).fit(X_train, y_train)\n",
|
||||
" clf_lasso = skl.Lasso(alpha=lmb).fit(X_train, y_train)\n",
|
||||
" yridge = clf_ridge.predict(X_test)\n",
|
||||
" ylasso = clf_lasso.predict(X_test)\n",
|
||||
" MSEPredictLasso[i] = MSE(y_test,ylasso)\n",
|
||||
" MSEPredictRidge[i] = MSE(y_test,yridge)\n",
|
||||
"#then plot the results\n",
|
||||
"plt.figure()\n",
|
||||
"plt.plot(np.log10(lambdas), MSEPredictRidge, 'r--', label = 'MSE Ridge Test')\n",
|
||||
"plt.plot(np.log10(lambdas), MSEPredictLasso, 'g--', label = 'MSE Lasso Test')\n",
|
||||
"plt.xlabel('log10(lambda)')\n",
|
||||
"plt.ylabel('MSE')\n",
|
||||
"plt.legend()\n",
|
||||
"plt.show()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<!-- --- end solution of exercise --- -->\n",
|
||||
"\n",
|
||||
"**e)**\n",
|
||||
"Finally, using **Scikit-Learn** or your own code, compute also the mean square error, a risk metric corresponding to the expected value of the squared (quadratic) error defined as"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"MSE(\\hat{y},\\hat{\\tilde{y}}) = \\frac{1}{n}\n",
|
||||
"\\sum_{i=0}^{n-1}(y_i-\\tilde{y}_i)^2,\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"and the $R^2$ score function.\n",
|
||||
"If $\\tilde{\\hat{y}}_i$ is the predicted value of the $i-th$ sample and $y_i$ is the corresponding true value, then the score $R^2$ is defined as"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"R^2(\\hat{y}, \\tilde{\\hat{y}}) = 1 - \\frac{\\sum_{i=0}^{n - 1} (y_i - \\tilde{y}_i)^2}{\\sum_{i=0}^{n - 1} (y_i - \\bar{y})^2},\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"where we have defined the mean value of $\\hat{y}$ as"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"$$\n",
|
||||
"\\bar{y} = \\frac{1}{n} \\sum_{i=0}^{n - 1} y_i.\n",
|
||||
"$$"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Discuss these quantities as functions of the variable $\\lambda$ in the Ridge and Lasso regression methods.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"<!-- --- begin solution of exercise --- -->\n",
|
||||
"**Solution.**\n",
|
||||
"These results can all be studied with the codes we have above. These scores are included in the codes above.\n",
|
||||
"\n",
|
||||
"<!-- --- end solution of exercise --- -->\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"<!-- --- end exercise --- -->\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"<!-- --- begin exercise --- -->\n",
|
||||
"\n",
|
||||
"## Exercise 2: Normalizing our data\n",
|
||||
"\n",
|
||||
"A much used approach before starting to train the data is to preprocess our\n",
|
||||
"data. Normally the data may need a rescaling and/or may be sensitive\n",
|
||||
"to extreme values. Scaling the data renders our inputs much more\n",
|
||||
"suitable for the algorithms we want to employ.\n",
|
||||
"\n",
|
||||
"**Scikit-Learn** has several functions which allow us to rescale the\n",
|
||||
"data, normally resulting in much better results in terms of various\n",
|
||||
"accuracy scores. The **StandardScaler** function in **Scikit-Learn**\n",
|
||||
"ensures that for each feature/predictor we study the mean value is\n",
|
||||
"zero and the variance is one (every column in the design/feature\n",
|
||||
"matrix). This scaling has the drawback that it does not ensure that\n",
|
||||
"we have a particular maximum or minimum in our data set. Another\n",
|
||||
"function included in **Scikit-Learn** is the **MinMaxScaler** which\n",
|
||||
"ensures that all features are exactly between $0$ and $1$. The\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"The **Normalizer** scales each data\n",
|
||||
"point such that the feature vector has a euclidean length of one. In other words, it\n",
|
||||
"projects a data point on the circle (or sphere in the case of higher dimensions) with a\n",
|
||||
"radius of 1. This means every data point is scaled by a different number (by the\n",
|
||||
"inverse of it’s length).\n",
|
||||
"This normalization is often used when only the direction (or angle) of the data matters,\n",
|
||||
"not the length of the feature vector.\n",
|
||||
"\n",
|
||||
"The **RobustScaler** works similarly to the StandardScaler in that it\n",
|
||||
"ensures statistical properties for each feature that guarantee that\n",
|
||||
"they are on the same scale. However, the RobustScaler uses the median\n",
|
||||
"and quartiles, instead of mean and variance. This makes the\n",
|
||||
"RobustScaler ignore data points that are very different from the rest\n",
|
||||
"(like measurement errors). These odd data points are also called\n",
|
||||
"outliers, and might often lead to trouble for other scaling\n",
|
||||
"techniques.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"It also common to split the data in a **training** set and a **testing** set. A typical split is to use $80\\%$ of the data for training and the rest\n",
|
||||
"for testing. This can be done as follows with our design matrix $\\boldsymbol{X}$ and data $\\boldsymbol{y}$ (remember to import **scikit-learn**)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"# split in training and test data\n",
|
||||
"X_train, X_test, y_train, y_test = train_test_split(X,y,test_size=0.2)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"Then we can use the standard scaler to scale our data as"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"scaler = StandardScaler()\n",
|
||||
"scaler.fit(X_train)\n",
|
||||
"X_train_scaled = scaler.transform(X_train)\n",
|
||||
"X_test_scaled = scaler.transform(X_test)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"In this exercise we want you to to compute the MSE for the training\n",
|
||||
"data and the test data as function of the complexity of a polynomial,\n",
|
||||
"that is the degree of a given polynomial. We want you also to compute the $R2$ score as function of the complexity of the model for both training data and test data. You should also run the calculation with and without scaling. \n",
|
||||
"\n",
|
||||
"One of \n",
|
||||
"the aims is to reproduce Figure 2.11 of [Hastie et al](https://github.com/CompPhysics/MLErasmus/blob/master/doc/Textbooks/elementsstat.pdf).\n",
|
||||
"We will also use Ridge and Lasso regression. \n",
|
||||
"\n",
|
||||
"\n",
|
||||
"Our data is defined by $x\\in [-3,3]$ with a total of for example $100$ data points."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"np.random.seed()\n",
|
||||
"n = 100\n",
|
||||
"maxdegree = 14\n",
|
||||
"# Make data set.\n",
|
||||
"x = np.linspace(-3, 3, n).reshape(-1, 1)\n",
|
||||
"y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"where $y$ is the function we want to fit with a given polynomial.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"**a)**\n",
|
||||
"Write a first code which sets up a design matrix $X$ defined by a fifth-order polynomial. Scale your data and split it in training and test data.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"<!-- --- begin solution of exercise --- -->\n",
|
||||
"**Solution.**"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 9,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import matplotlib.pyplot as plt\n",
|
||||
"import numpy as np\n",
|
||||
"from sklearn.linear_model import LinearRegression, Ridge, Lasso\n",
|
||||
"from sklearn.preprocessing import PolynomialFeatures\n",
|
||||
"from sklearn.model_selection import train_test_split\n",
|
||||
"from sklearn.pipeline import make_pipeline\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"np.random.seed(2018)\n",
|
||||
"n = 50\n",
|
||||
"maxdegree = 5\n",
|
||||
"# Make data set.\n",
|
||||
"x = np.linspace(-3, 3, n).reshape(-1, 1)\n",
|
||||
"y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape)\n",
|
||||
"TestError = np.zeros(maxdegree)\n",
|
||||
"TrainError = np.zeros(maxdegree)\n",
|
||||
"polydegree = np.zeros(maxdegree)\n",
|
||||
"x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2)\n",
|
||||
"scaler = StandardScaler()\n",
|
||||
"scaler.fit(X_train)\n",
|
||||
"x_train_scaled = scaler.transform(x_train)\n",
|
||||
"x_test_scaled = scaler.transform(x_test)\n",
|
||||
"\n",
|
||||
"for degree in range(maxdegree):\n",
|
||||
" model = make_pipeline(PolynomialFeatures(degree=degree), LinearRegression(fit_intercept=False))\n",
|
||||
" clf = model.fit(x_train_scale,y_train)\n",
|
||||
" y_fit = clf.predict(x_train_scaled)\n",
|
||||
" y_pred = clf.predict(x_test_scaled) \n",
|
||||
" polydegree[degree] = degree\n",
|
||||
" TestError[degree] = np.mean( np.mean((y_test - y_pred)**2) )\n",
|
||||
" TrainError[degree] = np.mean( np.mean((y_train - y_fit)**2) )\n",
|
||||
"\n",
|
||||
"plt.plot(polydegree, TestError, label='Test Error')\n",
|
||||
"plt.plot(polydegree, TrainError, label='Train Error')\n",
|
||||
"plt.legend()\n",
|
||||
"plt.show()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"<!-- --- end solution of exercise --- -->\n",
|
||||
"\n",
|
||||
"**b)**\n",
|
||||
"Perform an ordinary least squares and compute the means squared error and the $R2$ factor for the training data and the test data, with and without scaling.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"<!-- --- begin solution of exercise --- -->\n",
|
||||
"**Solution.**\n",
|
||||
"This requires a simple extension to the above code where you simply add a statement calling the $R2$ function included in the same code.\n",
|
||||
"\n",
|
||||
"<!-- --- end solution of exercise --- -->\n",
|
||||
"\n",
|
||||
"**c)**\n",
|
||||
"Add now a model which allows you to make polynomials up to degree $15$. Perform a standard OLS fitting of the training data and compute the MSE and $R2$ for the training and test data and plot both test and training data MSE and $R2$ as functions of the polynomial degree. Compare what you see with Figure 2.11 of Hastie et al. Comment your results. For which polynomial degree do you find an optimal MSE (smallest value)?\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"<!-- --- begin solution of exercise --- -->\n",
|
||||
"**Solution.**\n",
|
||||
"Here you simply need to change the degree of the polynomial in the above code to $n=15$.\n",
|
||||
"\n",
|
||||
"<!-- --- end solution of exercise --- -->\n",
|
||||
"\n",
|
||||
"**d)**\n",
|
||||
"Repeat part (2c) but now using Ridge regressions with various hyperparameters $\\lambda$. Make the same plots for the optimal $\\lambda$ value for each polynomial degree. Compare these results with those from the standard OLS approach.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"<!-- --- begin solution of exercise --- -->\n",
|
||||
"**Solution.**\n",
|
||||
"Here you need to add for example the same loop over the parameters $\\lambda$ as you did in the first exercise, that is add"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 10,
|
||||
"metadata": {
|
||||
"collapsed": false
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"nlambdas = 100\n",
|
||||
"MSEPredictRidge = np.zeros(nlambdas)\n",
|
||||
"lambdas = np.logspace(-4, 0, nlambdas)\n",
|
||||
"for i in range(nlambdas):\n",
|
||||
" lmb = lambdas[i]\n",
|
||||
" # add ridge\n",
|
||||
" clf_ridge = skl.Ridge(alpha=lmb).fit(X_train_scaled, y_train)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"The plotting functionality of the first exercise can be reused here as well.\n",
|
||||
"\n",
|
||||
"<!-- --- end solution of exercise --- -->\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"<!-- --- end exercise --- -->"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
Reference in New Issue
Block a user