added material about id3 also

doc/src/DecisionTrees/DecisionTrees.do.txt

@@ -34,6 +34,21 @@ A decision tree mainly contains of a _root node_, _interior nodes_,
 and _leaf nodes_ which are then connected by _branches_.
 
 
+!split
+===== General Features =====
+
+Decision trees classify instances by sorting top down.
+
+* A leaf provides the classification of the instance.
+* A node specifies a test of some attribute of the instance.
+* A branch corresponds to a possible values an attribute.
+* An instance is classified by starting at the root node of the tree, testing the attribute specified by this node, then moving down the tree branch corresponding to the value of the attribute in the given example.
+
+
+This processis then repeated for the subtree rooted at the new
+node.
+
+
 !split
 ===== How do we set it up? =====
 
@@ -364,7 +379,34 @@ s = -\sum_{k=1}^K p_{mk}\log{p_{mk}}.
 !split
 ===== Entropy and the ID3 algorithm =====
 
-More text and code to come here.
+ID3, learns decision trees by constructing
+them topdown, beginning with the question _which attribute should be tested at the root of the tree_?
+
+o Each instance attribute is evaluated using a statistical test to determine how well it alone classifies the training examples.
+o The best attribute is selected and used as the test at the root node of the tree.
+o A descendant of the root node is then created for each possible value of this attribute.
+o Training examples are sorted to the appropriate descendant node.
+o The entire process is then repeated using the training examples associated with each descendant node to select the best attribute to test at that point in the tree.
+o This forms a greedy search for an acceptable decision tree, in which the algorithm never backtracks to reconsider earlier choices. 
+
+The ID3 algorithm selects, which attribute to test at each node in the
+tree.
+
+We would like to select the attribute that is most useful for classifying
+examples.
+
+What is a good quantitative measure of the worth of an attribute?
+
+Information gain measures how well a given attribute separates the
+training examples according to their target classification.
+
+The ID3 algorithm uses this information gain measure to select among the candidate
+attributes at each step while growing the tree.
+
+!split
+=====  Implementing the ID3 Algorithm =====
+
+_more text to come here_, material presented during lecture Friday Oct 25.
 
 !split
 =====  Cancer Data again now with Decision Trees =====