lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

adding material to dim red

Browse Source
This commit is contained in:
mhjensen
2019-10-10 13:13:09 +02:00
parent f3d4490082
commit 88b667df81
55 changed files with 1955 additions and 3454 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/DimRed/html/DimRed.html
+47 -21
View File
@@ -6,9 +6,10 @@ Automatically generated HTML file from DocOnce source
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="description" content="Data Analysis and Machine Learning: Dimensionality Reduction">
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Preprocessing and Dimensionality Reduction">
<title>Data Analysis and Machine Learning: Dimensionality Reduction</title>
<title>Data Analysis and Machine Learning: Preprocessing and Dimensionality Reduction</title>
<style type="text/css">
@@ -69,14 +70,15 @@ div { text-align: justify; text-justify: inter-word; }
2,
None,
'___sec0'),
('Principal Component Analysis', 2, None, '___sec1'),
('PCA and scikit-learn', 2, None, '___sec2'),
('More on the PCA', 2, None, '___sec3'),
('Incremental PCA', 2, None, '___sec4'),
('Randomized PCA', 2, None, '___sec5'),
('Kernel PCA', 2, None, '___sec6'),
('LLE', 2, None, '___sec7'),
('Other techniques', 2, None, '___sec8')]}
('Preprocessing our data', 2, None, '___sec1'),
('Principal Component Analysis', 2, None, '___sec2'),
('PCA and scikit-learn', 2, None, '___sec3'),
('More on the PCA', 2, None, '___sec4'),
('Incremental PCA', 2, None, '___sec5'),
('Randomized PCA', 2, None, '___sec6'),
('Kernel PCA', 2, None, '___sec7'),
('LLE', 2, None, '___sec8'),
('Other techniques', 2, None, '___sec9')]}
end of tocinfo -->
<body>
@@ -102,7 +104,7 @@ MathJax.Hub.Config({
<center><h1>Data Analysis and Machine Learning: Dimensionality Reduction</h1></center> <!-- document title -->
<center><h1>Data Analysis and Machine Learning: Preprocessing and Dimensionality Reduction</h1></center> <!-- document title -->
<p>
<!-- author(s): Morten Hjorth-Jensen -->
@@ -118,7 +120,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Oct 26, 2018</h4></center> <!-- date -->
<center><h4>Oct 10, 2019</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -146,7 +148,31 @@ reduction techniques: the principal component analysis PCA, Kernel PCA, and Loca
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec1">Principal Component Analysis </h2>
<h2 id="___sec1">Preprocessing our data </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
<p>
Before we proceed however, we will discuss how to preprocess our
data. Till now and in connection with project 1 not met so many cases
where we are too sensitive to the scaling of our data. Normally the
data may need a rescaling and/or may be sensitive to extreme
values. Scaling the data renders our inputs much more suitable for the
algorithms we want to emply.
<p>
<b>Scikit-Learn</b> has several functions which allow us to rescale the data, normally resulting in much better results in terms of various accuracy scores. The <b>StandardScaler</b> function in <b>Scikit-Learn</b> ensures that for each feature/predictor we study the mean value is zero and the variance is zero (every column in the design/feature matrix).
This scaling has the drawback that it does not ensure that we have a particular maximum or minumum in our data set. Another function included in <b>Scikit-Learn</b> is the <b>MinMaxScaler</b> which ensures that all features are exactly between \( 0 \) and \( 1 \). The <b>Normalizer</b> function scale each column of the design matrix so that
</div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec2">Principal Component Analysis </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
@@ -182,7 +208,7 @@ X2D <span style="color: #666666">=</span> X_centered<span style="color: #666666"
<p>
<!-- !split -->
<h2 id="___sec2">PCA and scikit-learn </h2>
<h2 id="___sec3">PCA and scikit-learn </h2>
<p>
Scikit-Learn&#8217;s PCA class implements PCA using SVD decomposition just like we did before. The
@@ -213,7 +239,7 @@ More material to come here.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec3">More on the PCA </h2>
<h2 id="___sec4">More on the PCA </h2>
Instead of arbitrarily choosing the number of dimensions to reduce down to, it is generally preferable to
choose the number of dimensions that add up to a sufficiently large portion of the variance (e.g., 95%).
Unless, of course, you are reducing dimensionality for data visualization &#8212; in that case you will
@@ -241,7 +267,7 @@ X_reduced <span style="color: #666666">=</span> pca<span style="color: #666666">
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec4">Incremental PCA </h2>
<h2 id="___sec5">Incremental PCA </h2>
One problem with the preceding implementation of PCA is that it requires the whole training set to fit in
memory in order for the SVD algorithm to run. Fortunately, Incremental PCA (IPCA) algorithms have
been developed: you can split the training set into mini-batches and feed an IPCA algorithm one minibatch
@@ -251,7 +277,7 @@ instances arrive).
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec5">Randomized PCA </h2>
<h2 id="___sec6">Randomized PCA </h2>
<p>
Scikit-Learn offers yet another option to perform PCA, called Randomized PCA. This is a stochastic
@@ -266,7 +292,7 @@ previous algorithms when \( d \) is much smaller than \( n \).
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec6">Kernel PCA </h2>
<h2 id="___sec7">Kernel PCA </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
@@ -295,7 +321,7 @@ X_reduced <span style="color: #666666">=</span> rbf_pca<span style="color: #6666
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec7">LLE </h2>
<h2 id="___sec8">LLE </h2>
<p>
Locally Linear Embedding (LLE) is another very powerful nonlinear dimensionality reduction
@@ -307,7 +333,7 @@ these local relationships are best preserved (more details shortly).
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec8">Other techniques </h2>
<h2 id="___sec9">Other techniques </h2>
<p>
There are many other dimensionality reduction techniques, several of which are available in Scikit-Learn.
@@ -327,7 +353,7 @@ Here are some of the most popular:
<center style="font-size:80%">
<!-- copyright --> &copy; 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
<!-- copyright --> &copy; 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
</center>