update

doc/src/week45/week45.do.txt

@@ -317,6 +317,138 @@ the classical Principal Component Analysis (PCA) theorem with
 applications. This will be discussed later this semester ("week 43":"https://compphysics.github.io/MachineLearning/doc/pub/week43/html/week43-bs.html").
 
 
+!split
+===== Other ways of presenting a classification problem =====
+
+For a binary classifcation matrix, the so-called _confusion matrix_, is often used. It can also be extended to more catgeories/classes as well.
+The following quantities are then used
+o positive condition number $P$, which represents the number of real positive cases in the data (output one/true etc)
+o The condition negative number $N$ which is the number of negative cases (ouput zero/false etc)
+o The true positive number $TP$ which represents whether a positive test result has been correctly classified (the application of our trained model on a test data set)
+o The true negative $TN$ number which represents whether a negative test has been correctly classified
+o The false positive $FP$ number, a so-called type I error which tells us about the fraction of  positive test result which are wrongly classified
+o A false negative $FN$ number, a so-called type II error which, should be pretty obvious, indicates if a negative test has been wrongly classified.
+
+
+!split
+===== Combinations of classification results =====
+
+It is common in the literature to define various combinations the above numbers. The most commonly used are
+
+!bblock Sensitivity, recall, hit rate, or true positive rate $TPR$
+!bt
+\[
+{\displaystyle \mathrm {TPR} ={\frac {\mathrm {TP} }{\mathrm {P} }}={\frac {\mathrm {TP} }{\mathrm {TP} +\mathrm {FN} }}=1-\mathrm {FNR} }
+\]
+!et
+!eblock
+
+!bblock Specificity, selectivity or true negative rate $TNR$
+!bt
+\[
+{\displaystyle \mathrm {TNR} ={\frac {\mathrm {TN} }{\mathrm {N} }}={\frac {\mathrm {TN} }{\mathrm {TN} +\mathrm {FP} }}=1-\mathrm {FPR} }
+\]
+!et
+with the fall-out false positive rate
+!bt
+\[
+{\displaystyle \mathrm {FPR} ={\frac {\mathrm {FP} }{\mathrm {N} }}={\frac {\mathrm {FP} }{\mathrm {FP} +\mathrm {TN} }}=1-\mathrm {TNR} }
+\]
+!et
+!eblock
+
+!bblock Precision or positive predictive value $PPV$
+!bt
+\[
+{\displaystyle \mathrm {PPV} ={\frac {\mathrm {TP} }{\mathrm {TP} +\mathrm {FP} }}=1-\mathrm {FDR} }
+\]
+!et
+!eblock
+
+!bblock Negative predictive value $NPV$
+!bt
+\[
+{\displaystyle \mathrm {NPV} ={\frac {\mathrm {TN} }{\mathrm {TN} +\mathrm {FN} }}=1-\mathrm {FOR} }
+\]
+!et
+!eblock
+
+
+!bblock Miss rate or false negative rate $FNR$
+!bt
+\[
+{\displaystyle \mathrm {FNR} ={\frac {\mathrm {FN} }{\mathrm {P} }}={\frac {\mathrm {FN} }{\mathrm {FN} +\mathrm {TP} }} }
+\]
+!et
+!eblock
+
+
+!bblock False discovery rate $FDR$
+!bt
+\[
+{\displaystyle \mathrm {FDR} ={\frac {\mathrm {FP} }{\mathrm {FP} +\mathrm {TP} }}=1-\mathrm {PPV} }
+\]
+!et
+!eblock
+
+!bblock False omission rate $FOR$
+!bt
+\[
+{\displaystyle \mathrm {FOR} ={\frac {\mathrm {FN} }{\mathrm {FN} +\mathrm {TN} }}=1-\mathrm {NPV} }
+\]
+!et
+!eblock
+
+!split
+===== $F_1$ score =====
+
+In statistical analysis of binary classification, the F-score or
+F-measure is a measure of a test's accuracy. It is calculated from the
+precision and recall of the test, where the precision is the number of
+true positive results divided by the number of all positive results,
+including those not identified correctly, and the recall is the number
+of true positive results divided by the number of all samples that
+should have been identified as positive. Precision is also known as
+positive predictive value, and recall is also known as sensitivity in
+diagnostic binary classification.
+
+The F1 score is the harmonic mean of the precision and recall. It thus
+symmetrically represents both precision and recall in one metric.  The
+highest possible value of an F-score is 1.0, indicating perfect
+precision and recall, and the lowest possible value is 0, if either
+precision or recall are zero.
+
+
+It is defined as 
+!bt
+\[
+{\displaystyle \mathrm {F} _{1}=2\times {\frac {\mathrm {PPV} \times \mathrm {TPR} }{\mathrm {PPV} +\mathrm {TPR} }}={\frac {2\mathrm {TP} }{2\mathrm {TP} +\mathrm {FP} +\mathrm {FN} }}}
+\]
+!et
+
+!split
+===== ROC curve =====
+
+A receiver operating characteristic curve, or ROC curve, is a
+graphical plot that illustrates the performance of a binary classifier
+model at varying threshold values.
+
+The ROC curve is the plot of the true positive rate (TPR) against the false positive rate (FPR) at each threshold setting.
+See URL:"https://en.wikipedia.org/wiki/Receiver_operating_characteristic" for more discussions.
+
+
+
+!split
+===== Cumulative gain curve =====
+
+The cumulative gain curve is a performance evaluation used typically for binary classification problems.
+It plots the $TPR$ True Positive Rate or Sensitivity (which represents the 
+fraction of examples correctly classified
+against Predictive Positive Rate, which represents 
+the fraction of positively predicted examples.
+
+
+
 
 !split
 ===== Other measures in classification studies: Cancer Data  again =====
@@ -357,25 +489,6 @@ plt.show()
 !ec
 
 
-!split
-===== ROC curve =====
-
-A receiver operating characteristic curve, or ROC curve, is a
-graphical plot that illustrates the performance of a binary classifier
-model at varying threshold values.
-
-The ROC curve is the plot of the true positive rate (TPR) against the false positive rate (FPR) at each threshold setting.
-See URL:"https://en.wikipedia.org/wiki/Receiver_operating_characteristic" for more discussions.
-
-!split
-===== Cumulative gain curve =====
-
-The cumulative gain curve is a performance evaluation used typically for binary classification problems.
-It plots the $TPR$ True Positive Rate or Sensitivity (which represents the 
-fraction of examples correctly classified
-against Predictive Positive Rate, which represents 
-the fraction of positively predicted examples.
-
 
 
 !split