Forgot to update CART also

doc/src/DecisionTrees/DecisionTrees.do.txt

@@ -471,15 +471,60 @@ Two algorithms stand out in the set up of decision trees:
 o The CART (Classification And Regression Tree) algorithm for both classification and regression
 o The ID3 algorithm based on the computation of the information gain for classification
 
-We discuss both algorithms with applications here. The popular library _Scikit-Learn_ uses the CART algorithm. For classification problems you can use either the _gini_ index or the _entropy_ to split a tree in two branches.
+We discuss both algorithms with applications here. The popular library
+_Scikit-Learn_ uses the CART algorithm. For classification problems
+you can use either the _gini_ index or the _entropy_ to split a tree
+in two branches.
 
 !split
 ===== The CART algorithm for Classification =====
 
+For classification, the CART algorithm splits the data set in two subsets using a single feature $k$ and a threshold $t_k$.
+This could be for example a threshold set by a number below a certain circumference of a malign tumor.
+
+How do we find these two quantities?
+We search for the pair $(k,t_k)$ that produces the purest subset using for example the _gini_ factor $G$.
+The cost function it tries to minimize is then
+!bt
+\[
+C(k,t_k) = \frac{m_{\mathrm{left}}}{m}G_{\mathrm{left}}+ \frac{m_{\mathrm{right}}}{m}G_{\mathrm{right}},
+\]
+!et
+where $G_{\mathrm{left\left}}$ measures the impurity of the left/right subset  and $m_{\mathrm{left/right}}$
+ is the number of instances in the left/right subset
+
+Once it has successfully split the training set in two, it splits the subsets using the same logic, then the subsubsets
+and so on, recursively. It stops recursing once it reaches the maximum depth (defined by the
+$max\_depth$ hyperparameter), or if it cannot find a split that will reduce impurity. A few other
+hyperparameters control additional stopping conditions such as the $min\_samples\_split$,
+$min\_samples\_leaf$, $min\_weight\_fraction\_leaf$, and $max\_leaf\_nodes$.
 
 !split
 ===== The CART algorithm for Regression =====
 
+The CART algorithm for regression works is similar to the one for classification except that instead of trying to split the
+training set in a way that minimizes say the _gini_ or _entropy_ impurity, it now tries to split the training set in a way that minimizes our well-known mean-squared error (MSE). The cost function is now
+!bt
+\[
+C(k,t_k) = \frac{m_{\mathrm{left}}}{m}\mathrm{MSE}_{\mathrm{left}}+ \frac{m_{\mathrm{right}}}{m}\mathrm{MSE}_{\mathrm{right}}.
+\]
+!et
+Here the MSE for a specific node is defined as
+!bt
+\[
+\mathrm{MSE}_{\mathrm{node}}=\frac{1}{m_\mathrm{node}}\sum_{i\in \mathrm{node}}(\overline{y}_{\mathrm{node}}-y_i)^2,
+\]
+!et
+with
+!bt
+\[
+\overline{y}_{\mathrm{node}}=\frac{1}{m_\mathrm{node}}\sum_{i\in \mathrm{node}}y_i,
+\]
+!et
+the mean value of all observations in a specific node.
+
+Without any regularization, the regression task for decision trees, 
+just like for classification tasks, is  prone to overfitting.
 
 
 !split