added zoo data

doc/src/DecisionTrees/DecisionTrees.do.txt

@@ -370,7 +370,7 @@ For each value of $\alpha$  there corresponds a subtree $T \in T_0$ such that
 !et
 is as small as possible. Here $\overline{T}$ is 
 the number of terminal nodes of the tree $T$ , $R_m$ is the
-rectangle (i.e. the subset of predictor space)  corresponding to the $m$th terminal node.
+rectangle (i.e. the subset of predictor space)  corresponding to the $m$-th terminal node.
 
 The tuning parameter $\alpha$ controls a trade-off between the subtree’s
 com- plexity and its fit to the training data. When $\alpha = 0$, then the
@@ -395,9 +395,11 @@ subtree corresponding to $\alpha$.
 
 !bblock Building a Regression Tree
 o Use recursive binary splitting to grow a large tree on the training data, stopping only when each terminal node has fewer than some minimum number of observations.
-o  Apply cost complexity pruning to the large tree in order to obtain a sequence of best subtrees, as a function of $\alpha$.
-o Use for example $K$-fold cross-validation to choose $\alpha$. Divide the training observations into $K$ folds. For each $k=1,2,\dots,K$ we repeat Steps 1 and 2 on all but the $k$th fold of the training data. Then we valuate the mean squared prediction error on the data in the left-out $k$th fold, as a function of $\alpha$.
-o Then we average the results for each value of $alpha$, and pick $\alpha$ to minimize the average error.
+o Apply cost complexity pruning to the large tree in order to obtain a sequence of best subtrees, as a function of $\alpha$.
+o Use for example $K$-fold cross-validation to choose $\alpha$. Divide the training observations into $K$ folds. For each $k=1,2,\dots,K$ we: 
+  * repeat steps 1 and 2 on all but the $k$-th fold of the training data. 
+  * Then we valuate the mean squared prediction error on the data in the left-out $k$-th fold, as a function of $\alpha$.
+  * Finally  we average the results for each value of $alpha$, and pick $\alpha$ to minimize the average error.
 o Return the subtree from Step 2 that corresponds to the chosen value of $\alpha$. 
 !eblock
 
@@ -439,6 +441,31 @@ split, since these two approaches are more sensitive to node purity
 than is the classification error rate. 
 
 
+!split
+===== The zoo data =====
+!bc pycod 
+import pandas as pd
+import numpy as np
+from pprint import pprint
+from sklearn.tree import DecisionTreeClassifier
+
+#Import the dataset 
+dataset = pd.read_csv('data/zoo.csv')
+#We drop the animal names since this is not a good feature to split the data on
+#dataset=dataset.drop('animal_name',axis=1)
+#Split the data into a training and a testing set
+train_features = dataset.iloc[:80,:-1]
+test_features = dataset.iloc[80:,:-1]
+train_targets = dataset.iloc[:80,-1]
+test_targets = dataset.iloc[80:,-1]
+#Train the model
+tree = DecisionTreeClassifier(criterion = 'entropy').fit(train_features,train_targets)
+#Predict the classes of new, unseen data
+prediction = tree.predict(test_features)
+#Check the accuracy
+print("The prediction accuracy is: ",tree.score(test_features,test_targets)*100,"%")
+
+!ec
 
 !split
 ===== Pros and cons of trees, pros =====
@@ -553,3 +580,8 @@ Random_Forest_model = RandomForestClassifier(n_estimators=100,criterion="entropy
 #Cross validation
 accuracy = cross_validate(Random_Forest_model,X,Y,cv=10)['test_score']
 !ec
+
+
+!split 
+===== Boosting =====
+More material to come here.