updating PCA material

doc/src/DimRed/DimRed.do.txt

@@ -7,15 +7,21 @@ DATE: today
 ===== Reducing the number of degrees of freedom, overarching view  =====
 !bblock
 
-Many Machine Learning problems involve thousands or even millions of features for each training
-instance. Not only does this make training extremely slow, it can also make it much harder to find a good
-solution, as we will see. This problem is often referred to as the curse of dimensionality.
-Fortunately, in real-world problems, it is often possible to reduce the number of features considerably,
-turning an intractable problem into a tractable one.
+Many Machine Learning problems involve thousands or even millions of
+features for each training instance. Not only does this make training
+extremely slow, it can also make it much harder to find a good
+solution, as we will see. This problem is often referred to as the
+curse of dimensionality.  Fortunately, in real-world problems, it is
+often possible to reduce the number of features considerably, turning
+an intractable problem into a tractable one.
+
+Here we will discuss some of the most popular dimensionality reduction
+techniques: the principal component analysis PCA, Kernel PCA, and
+Locally Linear Embedding (LLE).  Furthermore, we will start by looking
+at some simple preprocessing of the data which allow us to rescale the
+data.
+
 
-Here we will discuss some of the most popular dimensionality
-reduction techniques: the principal component analysis PCA, Kernel PCA, and Locally Linear Embedding (LLE).
-Furthermore, we will start by looking at some simple preprocessing of the data which allow us to rescale the data. 
 !eblock
 
 
@@ -24,11 +30,11 @@ Furthermore, we will start by looking at some simple preprocessing of the data w
 !bblock
 
 Before we proceed however, we will discuss how to preprocess our
-data. Till now and in connection with our previous examples we have not met so many cases
-where we are too sensitive to the scaling of our data. Normally the
-data may need a rescaling and/or may be sensitive to extreme
-values. Scaling the data renders our inputs much more suitable for the
-algorithms we want to employ.
+data. Till now and in connection with our previous examples we have
+not met so many cases where we are too sensitive to the scaling of our
+data. Normally the data may need a rescaling and/or may be sensitive
+to extreme values. Scaling the data renders our inputs much more
+suitable for the algorithms we want to employ.
 
 _Scikit-Learn_ has several functions which allow us to rescale the
 data, normally resulting in much better results in terms of various
@@ -310,18 +316,36 @@ X_test_scaled = scaler.transform(X_test)
 logreg.fit(X_train_scaled, y_train)
 print("Test set accuracy scaled data: {:.2f}".format(logreg.score(X_test_scaled,y_test)))
 
-
-
-
-
 !ec
 
 
 
+#todo: add more text in order to explain what is done here, discuss the correlation matrix
+
+
+!split
+===== Basic ideas of the Principal Component Analysis (PCA) =====
+
+
+!split
+===== Introducing the Covariance and Correlation functions  =====
+
+!split
+===== Classical PCA Theorem   =====
+
+
+!split
+===== Prof of the PCA Theorem  =====
+
+
+
+
+
+
 !split
 ===== Getting started with PCA =====
 
-This material is being finalized, not yet ready.
+
 !bc pycod
 # Now add PCA
 from sklearn.decomposition import PCA
@@ -384,6 +408,7 @@ More material to come here.
 
 !split
 ===== More on the PCA =====
+
 Instead of arbitrarily choosing the number of dimensions to reduce down to, it is generally preferable to
 choose the number of dimensions that add up to a sufficiently large portion of the variance (e.g., 95%).
 Unless, of course, you are reducing dimensionality for data visualization — in that case you will
@@ -406,6 +431,7 @@ X_reduced = pca.fit_transform(X)
 
 !split
 ===== Incremental PCA =====
+
 One problem with the preceding implementation of PCA is that it requires the whole training set to fit in
 memory in order for the SVD algorithm to run. Fortunately, Incremental PCA (IPCA) algorithms have
 been developed: you can split the training set into mini-batches and feed an IPCA algorithm one minibatch