Another very useful piece of information is the explained variance ratio of each principal component, available via the \( explained\_variance\_ratio \) variable. It indicates the proportion of the dataset’s -variance that lies along the axis of each principal component. -More material to come here. +variance that lies along the axis of each principal component.
@@ -234,6 +235,7 @@ More material to come here.
-Instead of arbitrarily choosing the number of dimensions to reduce down to, it is generally preferable to -choose the number of dimensions that add up to a sufficiently large portion of the variance (e.g., 95%). -Unless, of course, you are reducing dimensionality for data visualization — in that case you will -generally want to reduce the dimensionality down to 2 or 3. -The following code computes PCA without reducing dimensionality, then computes the minimum number -of dimensions required to preserve 95% of the training set’s variance: +
-
pca = PCA()
-pca.fit(X)
-cumsum = np.cumsum(pca.explained_variance_ratio_)
-d = np.argmax(cumsum >= 0.95) + 1
--You could then set \( n\_components=d \) and run PCA again. However, there is a much better option: instead -of specifying the number of principal components you want to preserve, you can set \( n\_components \) to be -a float between 0.0 and 1.0, indicating the ratio of variance you wish to preserve: -
+
import matplotlib.pyplot as plt
+import numpy as np
+from sklearn.model_selection import train_test_split
+from sklearn.datasets import load_breast_cancer
+from sklearn.linear_model import LogisticRegression
+cancer = load_breast_cancer()
-
-pca = PCA(n_components=0.95)
-X_reduced = pca.fit_transform(X)
+X_train, X_test, y_train, y_test = train_test_split(cancer.data,cancer.target,random_state=0)
+print(X_train.shape)
+print(X_test.shape)
+
+logreg = LogisticRegression()
+logreg.fit(X_train, y_train)
+print("Test set accuracy from Logistic Regression: {:.2f}".format(logreg.score(X_test,y_test)))
+
+from sklearn.preprocessing import MinMaxScaler, StandardScaler
+scaler = StandardScaler()
+scaler.fit(X_train)
+X_train_scaled = scaler.transform(X_train)
+X_test_scaled = scaler.transform(X_test)
+
+logreg.fit(X_train_scaled, y_train)
+print("Test set accuracy scaled data: {:.2f}".format(logreg.score(X_test_scaled,y_test)))
+
+#thereafter we do a PCA with Scikit-learn
+from sklearn.decomposition import PCA
+pca = PCA(n_components = 2)
+X2D_train = pca.fit_transform(X_train_scaled)
+X2D_test = pca.fit_transform(X_test_scaled)
+
+logreg.fit(X2D_train,y_train)
+print("Test set accuracy scaled data: {:.2f}".format(logreg.score(X2D_test,y_test)))
@@ -230,6 +243,7 @@ X_reduced = pca
28
29
30
+ 31
»
diff --git a/doc/pub/DimRed/html/._DimRed-bs025.html b/doc/pub/DimRed/html/._DimRed-bs025.html
index dc43d79f6..213172eef 100644
--- a/doc/pub/DimRed/html/._DimRed-bs025.html
+++ b/doc/pub/DimRed/html/._DimRed-bs025.html
@@ -95,12 +95,13 @@ Automatically generated HTML file from DocOnce source
('The final step', 2, None, '___sec20'),
('Principal Component Analysis', 2, None, '___sec21'),
('PCA and scikit-learn', 2, None, '___sec22'),
- ('More on the PCA', 2, None, '___sec23'),
- ('Incremental PCA', 2, None, '___sec24'),
- ('Randomized PCA', 2, None, '___sec25'),
- ('Kernel PCA', 2, None, '___sec26'),
- ('LLE', 2, None, '___sec27'),
- ('Other techniques', 2, None, '___sec28')]}
+ ('Back to the Cancer Data', 2, None, '___sec23'),
+ ('More on the PCA', 2, None, '___sec24'),
+ ('Incremental PCA', 2, None, '___sec25'),
+ ('Randomized PCA', 2, None, '___sec26'),
+ ('Kernel PCA', 2, None, '___sec27'),
+ ('LLE', 2, None, '___sec28'),
+ ('Other techniques', 2, None, '___sec29')]}
end of tocinfo -->
@@ -161,12 +162,13 @@ MathJax.Hub.Config({
The final step
Principal Component Analysis
PCA and scikit-learn
- More on the PCA
- Incremental PCA
- Randomized PCA
- Kernel PCA
- LLE
- Other techniques
+ Back to the Cancer Data
+ More on the PCA
+ Incremental PCA
+ Randomized PCA
+ Kernel PCA
+ LLE
+ Other techniques
@@ -182,15 +184,33 @@ MathJax.Hub.Config({
-Incremental PCA
+More on the PCA
-One problem with the preceding implementation of PCA is that it requires the whole training set to fit in
-memory in order for the SVD algorithm to run. Fortunately, Incremental PCA (IPCA) algorithms have
-been developed: you can split the training set into mini-batches and feed an IPCA algorithm one minibatch
-at a time. This is useful for large training sets, and also to apply PCA online (i.e., on the fly, as new
-instances arrive).
+Instead of arbitrarily choosing the number of dimensions to reduce down to, it is generally preferable to
+choose the number of dimensions that add up to a sufficiently large portion of the variance (e.g., 95%).
+Unless, of course, you are reducing dimensionality for data visualization — in that case you will
+generally want to reduce the dimensionality down to 2 or 3.
+The following code computes PCA without reducing dimensionality, then computes the minimum number
+of dimensions required to preserve 95% of the training set’s variance:
+
+
+
pca = PCA()
+pca.fit(X)
+cumsum = np.cumsum(pca.explained_variance_ratio_)
+d = np.argmax(cumsum >= 0.95) + 1
+
+
+You could then set \( n\_components=d \) and run PCA again. However, there is a much better option: instead
+of specifying the number of principal components you want to preserve, you can set \( n\_components \) to be
+a float between 0.0 and 1.0, indicating the ratio of variance you wish to preserve:
+
+
+
+
pca = PCA(n_components=0.95)
+X_reduced = pca.fit_transform(X)
+
@@ -211,6 +231,7 @@ instances arrive).
28
29
30
+ 31
»
diff --git a/doc/pub/DimRed/html/._DimRed-bs026.html b/doc/pub/DimRed/html/._DimRed-bs026.html
index a4e1b973d..e3a14c74a 100644
--- a/doc/pub/DimRed/html/._DimRed-bs026.html
+++ b/doc/pub/DimRed/html/._DimRed-bs026.html
@@ -95,12 +95,13 @@ Automatically generated HTML file from DocOnce source
('The final step', 2, None, '___sec20'),
('Principal Component Analysis', 2, None, '___sec21'),
('PCA and scikit-learn', 2, None, '___sec22'),
- ('More on the PCA', 2, None, '___sec23'),
- ('Incremental PCA', 2, None, '___sec24'),
- ('Randomized PCA', 2, None, '___sec25'),
- ('Kernel PCA', 2, None, '___sec26'),
- ('LLE', 2, None, '___sec27'),
- ('Other techniques', 2, None, '___sec28')]}
+ ('Back to the Cancer Data', 2, None, '___sec23'),
+ ('More on the PCA', 2, None, '___sec24'),
+ ('Incremental PCA', 2, None, '___sec25'),
+ ('Randomized PCA', 2, None, '___sec26'),
+ ('Kernel PCA', 2, None, '___sec27'),
+ ('LLE', 2, None, '___sec28'),
+ ('Other techniques', 2, None, '___sec29')]}
end of tocinfo -->
@@ -161,12 +162,13 @@ MathJax.Hub.Config({
The final step
Principal Component Analysis
PCA and scikit-learn
- More on the PCA
- Incremental PCA
- Randomized PCA
- Kernel PCA
- LLE
- Other techniques
+ Back to the Cancer Data
+ More on the PCA
+ Incremental PCA
+ Randomized PCA
+ Kernel PCA
+ LLE
+ Other techniques
@@ -182,18 +184,14 @@ MathJax.Hub.Config({
-Randomized PCA
+Incremental PCA
-Scikit-Learn offers yet another option to perform PCA, called Randomized PCA. This is a stochastic
-algorithm that quickly finds an approximation of the first d principal components. Its computational
-complexity is \( O(m \times d^2)+O(d^3) \), instead of \( O(m \times n^2) + O(n^3) \), so it is dramatically faster than the
-previous algorithms when \( d \) is much smaller than \( n \).
-
-
-
@@ -214,6 +212,7 @@ previous algorithms when \( d \) is much smaller than \( n \).
+
-The kernel trick is a mathematical technique that implicitly maps instances into a -very high-dimensional space (called the feature space), enabling nonlinear classification and regression -with Support Vector Machines. Recall that a linear decision boundary in the high-dimensional feature -space corresponds to a complex nonlinear decision boundary in the original space. -It turns out that the same trick can be applied to PCA, making it possible to perform complex nonlinear -projections for dimensionality reduction. This is called Kernel PCA (kPCA). It is often good at -preserving clusters of instances after projection, or sometimes even unrolling datasets that lie close to a -twisted manifold. -For example, the following code uses Scikit-Learn’s KernelPCA class to perform kPCA with an -
+Scikit-Learn offers yet another option to perform PCA, called Randomized PCA. This is a stochastic +algorithm that quickly finds an approximation of the first d principal components. Its computational +complexity is \( O(m \times d^2)+O(d^3) \), instead of \( O(m \times n^2) + O(n^3) \), so it is dramatically faster than the +previous algorithms when \( d \) is much smaller than \( n \). - -
from sklearn.decomposition import KernelPCA
-rbf_pca = KernelPCA(n_components = 2, kernel="rbf", gamma=0.04)
-X_reduced = rbf_pca.fit_transform(X)
-
-Locally Linear Embedding (LLE) is another very powerful nonlinear dimensionality reduction -(NLDR) technique. It is a Manifold Learning technique that does not rely on projections like the previous -algorithms. In a nutshell, LLE works by first measuring how each training instance linearly relates to its -closest neighbors (c.n.), and then looking for a low-dimensional representation of the training set where -these local relationships are best preserved (more details shortly). +The kernel trick is a mathematical technique that implicitly maps instances into a +very high-dimensional space (called the feature space), enabling nonlinear classification and regression +with Support Vector Machines. Recall that a linear decision boundary in the high-dimensional feature +space corresponds to a complex nonlinear decision boundary in the original space. +It turns out that the same trick can be applied to PCA, making it possible to perform complex nonlinear +projections for dimensionality reduction. This is called Kernel PCA (kPCA). It is often good at +preserving clusters of instances after projection, or sometimes even unrolling datasets that lie close to a +twisted manifold. +For example, the following code uses Scikit-Learn’s KernelPCA class to perform kPCA with an +
+ + +
from sklearn.decomposition import KernelPCA
+rbf_pca = KernelPCA(n_components = 2, kernel="rbf", gamma=0.04)
+X_reduced = rbf_pca.fit_transform(X)
++
@@ -208,6 +228,7 @@ these local relationships are best preserved (more details shortly).
-There are many other dimensionality reduction techniques, several of which are available in Scikit-Learn. +Locally Linear Embedding (LLE) is another very powerful nonlinear dimensionality reduction +(NLDR) technique. It is a Manifold Learning technique that does not rely on projections like the previous +algorithms. In a nutshell, LLE works by first measuring how each training instance linearly relates to its +closest neighbors (c.n.), and then looking for a low-dimensional representation of the training set where +these local relationships are best preserved (more details shortly).
-Here are some of the most popular: - -
diff --git a/doc/pub/DimRed/html/._DimRed-bs030.html b/doc/pub/DimRed/html/._DimRed-bs030.html index 0e96ca026..aa8394e93 100644 --- a/doc/pub/DimRed/html/._DimRed-bs030.html +++ b/doc/pub/DimRed/html/._DimRed-bs030.html @@ -93,9 +93,9 @@ Automatically generated HTML file from DocOnce source ('Proof of the PCA Theorem', 2, None, '___sec18'), ('PCA Proof continued', 2, None, '___sec19'), ('The final step', 2, None, '___sec20'), - ('PCA and Scikit-Learn Functionality', 2, None, '___sec21'), - ('Principal Component Analysis', 2, None, '___sec22'), - ('PCA and scikit-learn', 2, None, '___sec23'), + ('Principal Component Analysis', 2, None, '___sec21'), + ('PCA and scikit-learn', 2, None, '___sec22'), + ('Back to the Cancer Data', 2, None, '___sec23'), ('More on the PCA', 2, None, '___sec24'), ('Incremental PCA', 2, None, '___sec25'), ('Randomized PCA', 2, None, '___sec26'), @@ -160,9 +160,9 @@ MathJax.Hub.Config({
Another very useful piece of information is the explained variance ratio of each principal component,
available via the \( explained\_variance\_ratio \) variable. It indicates the proportion of the dataset’s
-variance that lies along the axis of each principal component.
-More material to come here.
+variance that lies along the axis of each principal component.
+
+
+
Instead of arbitrarily choosing the number of dimensions to reduce down to, it is generally preferable to
@@ -1269,7 +1310,7 @@ X_reduced = pca.fit_transform(X)
One problem with the preceding implementation of PCA is that it requires the whole training set to fit in
@@ -1281,7 +1322,7 @@ instances arrive).
Scikit-Learn offers yet another option to perform PCA, called Randomized PCA. This is a stochastic
@@ -1295,7 +1336,7 @@ previous algorithms when \( d \) is much smaller than \( n \).
@@ -1321,7 +1362,7 @@ X_reduced = rbf_pca.fit_transform(X)
Locally Linear Embedding (LLE) is another very powerful nonlinear dimensionality reduction
@@ -1333,7 +1374,7 @@ these local relationships are best preserved (more details shortly).
There are many other dimensionality reduction techniques, several of which are available in Scikit-Learn.
diff --git a/doc/pub/DimRed/html/DimRed-solarized.html b/doc/pub/DimRed/html/DimRed-solarized.html
index 0925346bb..2933d3b74 100644
--- a/doc/pub/DimRed/html/DimRed-solarized.html
+++ b/doc/pub/DimRed/html/DimRed-solarized.html
@@ -115,12 +115,13 @@ div { text-align: justify; text-justify: inter-word; }
('The final step', 2, None, '___sec20'),
('Principal Component Analysis', 2, None, '___sec21'),
('PCA and scikit-learn', 2, None, '___sec22'),
- ('More on the PCA', 2, None, '___sec23'),
- ('Incremental PCA', 2, None, '___sec24'),
- ('Randomized PCA', 2, None, '___sec25'),
- ('Kernel PCA', 2, None, '___sec26'),
- ('LLE', 2, None, '___sec27'),
- ('Other techniques', 2, None, '___sec28')]}
+ ('Back to the Cancer Data', 2, None, '___sec23'),
+ ('More on the PCA', 2, None, '___sec24'),
+ ('Incremental PCA', 2, None, '___sec25'),
+ ('Randomized PCA', 2, None, '___sec26'),
+ ('Kernel PCA', 2, None, '___sec27'),
+ ('LLE', 2, None, '___sec28'),
+ ('Other techniques', 2, None, '___sec29')]}
end of tocinfo -->
Another very useful piece of information is the explained variance ratio of each principal component,
available via the \( explained\_variance\_ratio \) variable. It indicates the proportion of the dataset’s
-variance that lies along the axis of each principal component.
-More material to come here.
+variance that lies along the axis of each principal component.
+
+
+
+
Instead of arbitrarily choosing the number of dimensions to reduce down to, it is generally preferable to
@@ -1206,7 +1247,7 @@ X_reduced = pca.fit_transform(X)
One problem with the preceding implementation of PCA is that it requires the whole training set to fit in
@@ -1218,7 +1259,7 @@ instances arrive).
Scikit-Learn offers yet another option to perform PCA, called Randomized PCA. This is a stochastic
@@ -1233,7 +1274,7 @@ previous algorithms when \( d \) is much smaller than \( n \).
@@ -1262,7 +1303,7 @@ X_reduced = rbf_pca.fit_transform(X)
Locally Linear Embedding (LLE) is another very powerful nonlinear dimensionality reduction
@@ -1274,7 +1315,7 @@ these local relationships are best preserved (more details shortly).
There are many other dimensionality reduction techniques, several of which are available in Scikit-Learn.
diff --git a/doc/pub/DimRed/html/DimRed.html b/doc/pub/DimRed/html/DimRed.html
index 60f22749e..d6f859567 100644
--- a/doc/pub/DimRed/html/DimRed.html
+++ b/doc/pub/DimRed/html/DimRed.html
@@ -120,12 +120,13 @@ div { text-align: justify; text-justify: inter-word; }
('The final step', 2, None, '___sec20'),
('Principal Component Analysis', 2, None, '___sec21'),
('PCA and scikit-learn', 2, None, '___sec22'),
- ('More on the PCA', 2, None, '___sec23'),
- ('Incremental PCA', 2, None, '___sec24'),
- ('Randomized PCA', 2, None, '___sec25'),
- ('Kernel PCA', 2, None, '___sec26'),
- ('LLE', 2, None, '___sec27'),
- ('Other techniques', 2, None, '___sec28')]}
+ ('Back to the Cancer Data', 2, None, '___sec23'),
+ ('More on the PCA', 2, None, '___sec24'),
+ ('Incremental PCA', 2, None, '___sec25'),
+ ('Randomized PCA', 2, None, '___sec26'),
+ ('Kernel PCA', 2, None, '___sec27'),
+ ('LLE', 2, None, '___sec28'),
+ ('Other techniques', 2, None, '___sec29')]}
end of tocinfo -->
Another very useful piece of information is the explained variance ratio of each principal component,
available via the \( explained\_variance\_ratio \) variable. It indicates the proportion of the dataset’s
-variance that lies along the axis of each principal component.
-More material to come here.
+variance that lies along the axis of each principal component.
+
+
+
+
Instead of arbitrarily choosing the number of dimensions to reduce down to, it is generally preferable to
@@ -1211,7 +1252,7 @@ X_reduced = pca
One problem with the preceding implementation of PCA is that it requires the whole training set to fit in
@@ -1223,7 +1264,7 @@ instances arrive).
Scikit-Learn offers yet another option to perform PCA, called Randomized PCA. This is a stochastic
@@ -1238,7 +1279,7 @@ previous algorithms when \( d \) is much smaller than \( n \).
@@ -1267,7 +1308,7 @@ X_reduced = rbf_pcaLLE
+
Locally Linear Embedding (LLE) is another very powerful nonlinear dimensionality reduction
@@ -1279,7 +1320,7 @@ these local relationships are best preserved (more details shortly).
There are many other dimensionality reduction techniques, several of which are available in Scikit-Learn.
diff --git a/doc/pub/DimRed/ipynb/DimRed.ipynb b/doc/pub/DimRed/ipynb/DimRed.ipynb
index c76867fcf..d42b6364e 100644
--- a/doc/pub/DimRed/ipynb/DimRed.ipynb
+++ b/doc/pub/DimRed/ipynb/DimRed.ipynb
@@ -1431,8 +1431,57 @@
"Another very useful piece of information is the explained variance ratio of each principal component,\n",
"available via the $explained\\_variance\\_ratio$ variable. It indicates the proportion of the dataset’s\n",
"variance that lies along the axis of each principal component. \n",
- "More material to come here.\n",
"\n",
+ "## Back to the Cancer Data\n",
+ "We can now repeat the above but applied to real data, in this case our breat cancer data."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": 15,
+ "metadata": {
+ "collapsed": false
+ },
+ "outputs": [],
+ "source": [
+ "import matplotlib.pyplot as plt\n",
+ "import numpy as np\n",
+ "from sklearn.model_selection import train_test_split \n",
+ "from sklearn.datasets import load_breast_cancer\n",
+ "from sklearn.linear_model import LogisticRegression\n",
+ "cancer = load_breast_cancer()\n",
+ "\n",
+ "X_train, X_test, y_train, y_test = train_test_split(cancer.data,cancer.target,random_state=0)\n",
+ "print(X_train.shape)\n",
+ "print(X_test.shape)\n",
+ "\n",
+ "logreg = LogisticRegression()\n",
+ "logreg.fit(X_train, y_train)\n",
+ "print(\"Test set accuracy from Logistic Regression: {:.2f}\".format(logreg.score(X_test,y_test)))\n",
+ "\n",
+ "from sklearn.preprocessing import MinMaxScaler, StandardScaler\n",
+ "scaler = StandardScaler()\n",
+ "scaler.fit(X_train)\n",
+ "X_train_scaled = scaler.transform(X_train)\n",
+ "X_test_scaled = scaler.transform(X_test)\n",
+ "\n",
+ "logreg.fit(X_train_scaled, y_train)\n",
+ "print(\"Test set accuracy scaled data: {:.2f}\".format(logreg.score(X_test_scaled,y_test)))\n",
+ "\n",
+ "#thereafter we do a PCA with Scikit-learn\n",
+ "from sklearn.decomposition import PCA\n",
+ "pca = PCA(n_components = 2)\n",
+ "X2D_train = pca.fit_transform(X_train_scaled)\n",
+ "X2D_test = pca.fit_transform(X_test_scaled)\n",
+ "\n",
+ "logreg.fit(X2D_train,y_train)\n",
+ "print(\"Test set accuracy scaled data: {:.2f}\".format(logreg.score(X2D_test,y_test)))"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
"## More on the PCA\n",
"\n",
"Instead of arbitrarily choosing the number of dimensions to reduce down to, it is generally preferable to\n",
@@ -1445,7 +1494,7 @@
},
{
"cell_type": "code",
- "execution_count": 15,
+ "execution_count": 16,
"metadata": {
"collapsed": false
},
@@ -1468,7 +1517,7 @@
},
{
"cell_type": "code",
- "execution_count": 16,
+ "execution_count": 17,
"metadata": {
"collapsed": false
},
@@ -1515,7 +1564,7 @@
},
{
"cell_type": "code",
- "execution_count": 17,
+ "execution_count": 18,
"metadata": {
"collapsed": false
},
diff --git a/doc/pub/DimRed/ipynb/ipynb-DimRed-src.tar.gz b/doc/pub/DimRed/ipynb/ipynb-DimRed-src.tar.gz
index 3b0146384..2b7fcb7fc 100644
Binary files a/doc/pub/DimRed/ipynb/ipynb-DimRed-src.tar.gz and b/doc/pub/DimRed/ipynb/ipynb-DimRed-src.tar.gz differ
diff --git a/doc/pub/DimRed/pdf/DimRed-minted.pdf b/doc/pub/DimRed/pdf/DimRed-minted.pdf
index a486cd0e6..acb39efc5 100644
Binary files a/doc/pub/DimRed/pdf/DimRed-minted.pdf and b/doc/pub/DimRed/pdf/DimRed-minted.pdf differ
diff --git a/doc/src/DimRed/DimRed.do.txt b/doc/src/DimRed/DimRed.do.txt
index 163230d7f..d8ad23a89 100644
--- a/doc/src/DimRed/DimRed.do.txt
+++ b/doc/src/DimRed/DimRed.do.txt
@@ -948,7 +948,45 @@ pca.components_.T[:, 0].
Another very useful piece of information is the explained variance ratio of each principal component,
available via the $explained\_variance\_ratio$ variable. It indicates the proportion of the dataset’s
variance that lies along the axis of each principal component.
-More material to come here.
+
+!split
+===== Back to the Cancer Data =====
+We can now repeat the above but applied to real data, in this case our breat cancer data.
+!bc pycod
+import matplotlib.pyplot as plt
+import numpy as np
+from sklearn.model_selection import train_test_split
+from sklearn.datasets import load_breast_cancer
+from sklearn.linear_model import LogisticRegression
+cancer = load_breast_cancer()
+
+X_train, X_test, y_train, y_test = train_test_split(cancer.data,cancer.target,random_state=0)
+print(X_train.shape)
+print(X_test.shape)
+
+logreg = LogisticRegression()
+logreg.fit(X_train, y_train)
+print("Test set accuracy from Logistic Regression: {:.2f}".format(logreg.score(X_test,y_test)))
+
+from sklearn.preprocessing import MinMaxScaler, StandardScaler
+scaler = StandardScaler()
+scaler.fit(X_train)
+X_train_scaled = scaler.transform(X_train)
+X_test_scaled = scaler.transform(X_test)
+
+logreg.fit(X_train_scaled, y_train)
+print("Test set accuracy scaled data: {:.2f}".format(logreg.score(X_test_scaled,y_test)))
+
+#thereafter we do a PCA with Scikit-learn
+from sklearn.decomposition import PCA
+pca = PCA(n_components = 2)
+X2D_train = pca.fit_transform(X_train_scaled)
+X2D_test = pca.fit_transform(X_test_scaled)
+
+logreg.fit(X2D_train,y_train)
+print("Test set accuracy scaled data: {:.2f}".format(logreg.score(X2D_test,y_test)))
+!ec
+
!split
===== More on the PCA =====
diff --git a/doc/src/DimRed/PCAcancer.py b/doc/src/DimRed/PCAcancer.py
index c0b2f2219..b2e267cf4 100644
--- a/doc/src/DimRed/PCAcancer.py
+++ b/doc/src/DimRed/PCAcancer.py
@@ -14,7 +14,6 @@ correlation_matrix = cancerpd.corr().round(1)
# use the heatmap function from seaborn to plot the correlation matrix
# annot = True to print the values inside the square
sns.heatmap(data=correlation_matrix, annot=True)
-plt.show()
EigValues, EigVectors = np.linalg.eig(correlation_matrix)
print(EigValues)
@@ -35,6 +34,11 @@ X_test_scaled = scaler.transform(X_test)
logreg.fit(X_train_scaled, y_train)
print("Test set accuracy scaled data: {:.2f}".format(logreg.score(X_test_scaled,y_test)))
+#thereafter we do a PCA with Scikit-learn
+from sklearn.decomposition import PCA
+pca = PCA(n_components = 2)
+X2D_train = pca.fit_transform(X_train_scaled)
+X2D_test = pca.fit_transform(X_test_scaled)
-
-
+logreg.fit(X2D_train,y_train)
+print("Test set accuracy scaled data: {:.2f}".format(logreg.score(X2D_test,y_test)))
More on the PCA
+Back to the Cancer Data
+We can now repeat the above but applied to real data, in this case our breat cancer data.
+import matplotlib.pyplot as plt
+import numpy as np
+from sklearn.model_selection import train_test_split
+from sklearn.datasets import load_breast_cancer
+from sklearn.linear_model import LogisticRegression
+cancer = load_breast_cancer()
+
+X_train, X_test, y_train, y_test = train_test_split(cancer.data,cancer.target,random_state=0)
+print(X_train.shape)
+print(X_test.shape)
+
+logreg = LogisticRegression()
+logreg.fit(X_train, y_train)
+print("Test set accuracy from Logistic Regression: {:.2f}".format(logreg.score(X_test,y_test)))
+
+from sklearn.preprocessing import MinMaxScaler, StandardScaler
+scaler = StandardScaler()
+scaler.fit(X_train)
+X_train_scaled = scaler.transform(X_train)
+X_test_scaled = scaler.transform(X_test)
+
+logreg.fit(X_train_scaled, y_train)
+print("Test set accuracy scaled data: {:.2f}".format(logreg.score(X_test_scaled,y_test)))
+
+#thereafter we do a PCA with Scikit-learn
+from sklearn.decomposition import PCA
+pca = PCA(n_components = 2)
+X2D_train = pca.fit_transform(X_train_scaled)
+X2D_test = pca.fit_transform(X_test_scaled)
+
+logreg.fit(X2D_train,y_train)
+print("Test set accuracy scaled data: {:.2f}".format(logreg.score(X2D_test,y_test)))
+
More on the PCA
Incremental PCA
+Incremental PCA
Randomized PCA
+Randomized PCA
Kernel PCA
+Kernel PCA
LLE
+LLE
Other techniques
+Other techniques
-More on the PCA
+Back to the Cancer Data
+We can now repeat the above but applied to real data, in this case our breat cancer data.
+import matplotlib.pyplot as plt
+import numpy as np
+from sklearn.model_selection import train_test_split
+from sklearn.datasets import load_breast_cancer
+from sklearn.linear_model import LogisticRegression
+cancer = load_breast_cancer()
+
+X_train, X_test, y_train, y_test = train_test_split(cancer.data,cancer.target,random_state=0)
+print(X_train.shape)
+print(X_test.shape)
+
+logreg = LogisticRegression()
+logreg.fit(X_train, y_train)
+print("Test set accuracy from Logistic Regression: {:.2f}".format(logreg.score(X_test,y_test)))
+
+from sklearn.preprocessing import MinMaxScaler, StandardScaler
+scaler = StandardScaler()
+scaler.fit(X_train)
+X_train_scaled = scaler.transform(X_train)
+X_test_scaled = scaler.transform(X_test)
+
+logreg.fit(X_train_scaled, y_train)
+print("Test set accuracy scaled data: {:.2f}".format(logreg.score(X_test_scaled,y_test)))
+
+#thereafter we do a PCA with Scikit-learn
+from sklearn.decomposition import PCA
+pca = PCA(n_components = 2)
+X2D_train = pca.fit_transform(X_train_scaled)
+X2D_test = pca.fit_transform(X_test_scaled)
+
+logreg.fit(X2D_train,y_train)
+print("Test set accuracy scaled data: {:.2f}".format(logreg.score(X2D_test,y_test)))
+
+
+More on the PCA
-Incremental PCA
+Incremental PCA
-Randomized PCA
+Randomized PCA
-Kernel PCA
+Kernel PCA
-LLE
+LLE
-Other techniques
+Other techniques
-More on the PCA
+Back to the Cancer Data
+We can now repeat the above but applied to real data, in this case our breat cancer data.
+import matplotlib.pyplot as plt
+import numpy as np
+from sklearn.model_selection import train_test_split
+from sklearn.datasets import load_breast_cancer
+from sklearn.linear_model import LogisticRegression
+cancer = load_breast_cancer()
+
+X_train, X_test, y_train, y_test = train_test_split(cancer.data,cancer.target,random_state=0)
+print(X_train.shape)
+print(X_test.shape)
+
+logreg = LogisticRegression()
+logreg.fit(X_train, y_train)
+print("Test set accuracy from Logistic Regression: {:.2f}".format(logreg.score(X_test,y_test)))
+
+from sklearn.preprocessing import MinMaxScaler, StandardScaler
+scaler = StandardScaler()
+scaler.fit(X_train)
+X_train_scaled = scaler.transform(X_train)
+X_test_scaled = scaler.transform(X_test)
+
+logreg.fit(X_train_scaled, y_train)
+print("Test set accuracy scaled data: {:.2f}".format(logreg.score(X_test_scaled,y_test)))
+
+#thereafter we do a PCA with Scikit-learn
+from sklearn.decomposition import PCA
+pca = PCA(n_components = 2)
+X2D_train = pca.fit_transform(X_train_scaled)
+X2D_test = pca.fit_transform(X_test_scaled)
+
+logreg.fit(X2D_train,y_train)
+print("Test set accuracy scaled data: {:.2f}".format(logreg.score(X2D_test,y_test)))
+
+
+More on the PCA
-Incremental PCA
+Incremental PCA
-Randomized PCA
+Randomized PCA
-Kernel PCA
+Kernel PCA
LLE
-Other techniques
+Other techniques