update

doc/src/week44/week44.do.txt

@@ -26,6 +26,36 @@ o Clustering and PCA, see Geron's chapter 8 and "Lecture notes":"https://compphy
 For the principal component analysis, 
 see slides from "week 43":"https://compphysics.github.io/MachineLearning/doc/pub/week43/html/week43-reveal.html", in particular from slide 28 and forward
 
+!split
+=====  A kind of Bird's view  on PCA =====
+
+
+_Why do we maximize variance during Principal Component Analysis?_
+
+Variance is a measure of the *variability* of the data you
+have. Potentially the number of components is infinite, so you want to "squeeze" the most
+information in each component of the finite set you build.
+
+If, to exaggerate, you were to select a single principal component,
+you would want it to account for the most variability possible: hence
+the search for maximum variance, so that the one component collects
+the most "uniqueness" from the data set.
+
+
+Maximizing the component vector variances is the same as maximizing
+the 'uniqueness' of those vectors. The vectors are as distant
+from each other as possible (orthogonal to each other).
+
+
+Take for example a situation where you have 2 lines that are
+orthogonal in a 3D space. You can capture the environment much more
+completely with those orthogonal lines than 2 lines that are parallel
+(or nearly parallel). When applied to very high dimensional states
+using very few vectors, this becomes a much more important
+relationship among the vectors to maintain. In a linear algebra sense
+you want independent rows to be produced by PCA, otherwise some of
+those rows will be redundant.
+
 !split
 ===== Thursday: Clustering and Unsupervised Learning =====
 
@@ -65,13 +95,10 @@ can think of these mappings as an encoder $k = C(i)$, which assigns the $i$-th
 data-point $\bf x_i$ to the $k$-th cluster.
 
 $k$-means algorithm in words:
-o We start with guesses / random initializations of our $k$ cluster centers /
-  centroids
+o We start with guesses / random initializations of our $k$ cluster centers/centroids
 o For each centroid the points that are most similar are identified
-o Then we move / replace each centroid with a coordinate average of all the
-  points that were assigned to that centroid.
-o Iterate this points 2, 3) until the centroids no longer move (to some
-  tolerance)
+o Then we move / replace each centroid with a coordinate average of all the points that were assigned to that centroid.
+o Iterate 2-3 until the centroids no longer move (to some tolerance)
 
 !split
 ===== Basic Math of the $k$-means Algorithm =====
@@ -150,14 +177,8 @@ Now we have all the pieces necessary to formally revisit the $k$-means algorithm
 
 The $k$-means clustering algorithm goes as follows 
 
-o For a given cluster assignment $C$, and $k$ cluster means
-  $\left\{m_1, \cdots, m_k\right\}$. We minimize the total cluster variance with respect to
-  the cluster means $\{m_k\}$ yielding the means of the currently assigned
-  clusters.
-o Given a current set of $k$ means $\{m_k\}$ the total cluster variance is
-  minimized by assigning each observation to the closest (current) cluster mean.
-  That is $$C(i) = \underset{1\leq k\leq K}{\mathrm{argmin}}
-  ||\bm{x_i} - \bm{m_k}||^2$$
+o For a given cluster assignment $C$, and $k$ cluster means $\left\{m_1, \cdots, m_k\right\}$. We minimize the total cluster variance with respect to the cluster means $\{m_k\}$ yielding the means of the currently assigned clusters.
+o Given a current set of $k$ means $\{m_k\}$ the total cluster variance is minimized by assigning each observation to the closest (current) cluster mean. That is $$C(i) = \underset{1\leq k\leq K}{\mathrm{argmin}} ||\bm{x_i} - \bm{m_k}||^2$$
 o Steps 1 and 2 are repeated until the assignments do not change.
 
 
@@ -165,17 +186,11 @@ o Steps 1 and 2 are repeated until the assignments do not change.
 ===== Summarizing  =====
 
 
-o Before we start we specify a number $k$ which is the number of clusters we
-  want to try to separate our data into.
-o We initially choose $k$ random data points in our data as our initial
-  centroids, *or means* (this is where the name comes from).
-o Assign each data point to their closest centroid, based on the squared
-  Euclidean distance.
-o For each of the $k$ cluster we update the centroid by calculating new mean
-  values for all the data points in the cluster.
-o Iteratively minimize the within cluster scatter by performing steps (3, 4)
-  until the new assignments stop changing (can be to some tolerance) or
-  until a maximum number of iterations have passed.
+o Before we start we specify a number $k$ which is the number of clusters we want to try to separate our data into.
+o We initially choose $k$ random data points in our data as our initial centroids, *or means* (this is where the name comes from).
+o Assign each data point to their closest centroid, based on the squared Euclidean distance.
+o For each of the $k$ cluster we update the centroid by calculating new mean values for all the data points in the cluster.
+o Iteratively minimize the within cluster scatter by performing steps (3, 4) until the new assignments stop changing (can be to some tolerance) or until a maximum number of iterations have passed.
 
 
 !split