update week44

doc/src/week44/Programs/simpletree.py

@@ -0,0 +1,7 @@
+from sklearn.datasets import load_iris
+from sklearn import tree
+X, y = load_iris(return_X_y=True)
+tree_clf = tree.DecisionTreeClassifier()
+tree_clf = tree_clf.fit(X, y)
+# and then plot	the tree
+tree.plot_tree(tree_clf)

doc/src/week44/week44.do.txt

@@ -6,7 +6,7 @@ DATE: today
 !split
 ===== Overview of week 44 =====
 
-* Thursday: Wrapping up PCA from last week and basics of decision trees, classification and regression algorithms 
+* "Thursday: Wrapping up PCA from last week and basics of decision trees, classification and regression algorithms with video of lecture":"https://www.uio.no/studier/emner/matnat/fys/FYS-STK4155/h20/forelesningsvideoer/LectureOctober29.mp4?vrtx=view-as-webpage" 
 * Friday: Decision trees, voting models and bagging
 
 
@@ -16,7 +16,7 @@ Geron's chapter 6 covers decision trees while ensemble models, voting and baggin
 !split
 ===== Thursday =====
 
-Overview video, aims and motivations. 
+
 
 !split
 ===== Decision trees, overarching aims  =====
@@ -313,8 +313,11 @@ way to do just this. Rather than considering every possible subtree,
 we consider a sequence of trees indexed by a nonnegative tuning
 parameter $\alpha$.
 
+Read more at the following "Scikit-Learn link on pruning":"https://scikit-learn.org/stable/auto_examples/tree/plot_cost_complexity_pruning.html#sphx-glr-auto-examples-tree-plot-cost-complexity-pruning-py".
+
 !split
 ===== Cost complexity pruning =====
+
 For each value of $\alpha$  there corresponds a subtree $T \in T_0$ such that
 !bt
 \[
@@ -326,7 +329,7 @@ the number of terminal nodes of the tree $T$ , $R_m$ is the
 rectangle (i.e. the subset of predictor space)  corresponding to the $m$-th terminal node.
 
 The tuning parameter $\alpha$ controls a trade-off between the subtree’s
-com- plexity and its fit to the training data. When $\alpha = 0$, then the
+complexity and its fit to the training data. When $\alpha = 0$, then the
 subtree $T$ will simply equal $T_0$, 
 because then the above equation just measures the
 training error. 
@@ -347,6 +350,7 @@ subtree corresponding to $\alpha$.
 ===== Schematic Regression Procedure =====
 
 !bblock Building a Regression Tree
+
 o Use recursive binary splitting to grow a large tree on the training data, stopping only when each terminal node has fewer than some minimum number of observations.
 o Apply cost complexity pruning to the large tree in order to obtain a sequence of best subtrees, as a function of $\alpha$.
 o Use for example $K$-fold cross-validation to choose $\alpha$. Divide the training observations into $K$ folds. For each $k=1,2,\dots,K$ we: 
@@ -502,6 +506,38 @@ cmd = 'dot -Tpng DataFiles/moons.dot -o DataFiles/moons.png'
 os.system(cmd)
 !ec
 
+!split
+===== Other ways of visualizing the trees =====
+
+_Scikit-Learn_ has also another way to visualize the trees which is very useful, here with the Iris data.
+
+!bc pycod
+from sklearn.datasets import load_iris
+from sklearn import tree
+X, y = load_iris(return_X_y=True)
+tree_clf = tree.DecisionTreeClassifier()
+tree_clf = tree_clf.fit(X, y)
+# and then plot the tree
+tree.plot_tree(tree_clf) 
+!ec
+
+!split
+===== Printing out as text =====
+
+Alternatively, the tree can also be exported in textual format with the function exporttext.
+This method doesn’t require the installation of external libraries and is more compact:
+
+!bc pycod
+from sklearn.datasets import load_iris
+from sklearn.tree import DecisionTreeClassifier
+from sklearn.tree import export_text
+iris = load_iris()
+decision_tree = DecisionTreeClassifier(random_state=0, max_depth=2)
+decision_tree = decision_tree.fit(iris.data, iris.target)
+r = export_text(decision_tree, feature_names=iris['feature_names'])
+print(r)
+!ec
+
 
 !split
 ===== Algorithms for Setting up Decision Trees =====