diff --git a/doc/src/week38/week38.do.txt b/doc/src/week38/week38.do.txt index 590a65fa7..ee7d4d4a5 100644 --- a/doc/src/week38/week38.do.txt +++ b/doc/src/week38/week38.do.txt @@ -1126,15 +1126,25 @@ f(y_i\vert x_i)=\beta_0+\beta_1 x_i. \] !et -This expression implies however that $f(y_i\vert x_i)$ could take any value from minus infinity to plus infinity. If we however let $f(y\vert y)$ be represented by the mean value, the above example shows us that we can constain to be between zero and one, that is we have $0 \le f(y_i\vert x_i) \le 1$. Looking at our last curve we see also that it has an S-shaped form. This leads us to a very popular model for the function $f$, namely the so-called Sigmoid function or logistic model. We will consider this function as representing the probability for finding a value of $y_i$ with a given $x_i$. +This expression implies however that $f(y_i\vert x_i)$ could take any +value from minus infinity to plus infinity. If we however let +$f(y\vert y)$ be represented by the mean value, the above example +shows us that we can constrain the function to take values between +zero and one, that is we have $0 \le f(y_i\vert x_i) \le 1$. Looking +at our last curve we see also that it has an S-shaped form. This leads +us to a very popular model for the function $f$, namely the so-called +Sigmoid function or logistic model. We will consider this function as +representing the probability for finding a value of $y_i$ with a given +$x_i$. !split ===== The logistic function ===== -The perceptron is an example of a ``hard classification'' model. We +Another widely studied model, is the so-called +perceptron model, which is an example of a ``hard classification'' model. We will encounter this model when we discuss neural networks as well. Each datapoint is deterministically assigned to a category (i.e -$y_i=0$ or $y_i=1$). In many cases, it is favorable to have a ``soft'' +$y_i=0$ or $y_i=1$). In many cases, and the coronary heart disease data forms one of many such examples, it is favorable to have a ``soft'' classifier that outputs the probability of a given category rather than a single value. For example, given $x_i$, the classifier outputs the probability of being in a category $k$. Logistic regression @@ -1344,14 +1354,14 @@ p(\hat{\beta}\hat{x})=\frac{ \exp{(\beta_0+\beta_1x_1+\beta_2x_2+\dots+\beta_px_ Till now we have mainly focused on two classes, the so-called binary system. Suppose we wish to extend to $K$ classes. Let us for the sake -of simplicity assume we have only two predictors. We have then -following model +of simplicity assume we have only two predictors. We have then following model !bt \[ \log{\frac{p(C=1\vert x)}{p(K\vert x)}} = \beta_{10}+\beta_{11}x_1, \] !et +and !bt \[ \log{\frac{p(C=2\vert x)}{p(K\vert x)}} = \beta_{20}+\beta_{21}x_1, @@ -1405,14 +1415,14 @@ descent method. Newton's method and gradient descent methods are discussed in the material on "optimization methods":"https://compphysics.github.io/MachineLearning/doc/pub/Splines/html/Splines-bs.html". -This will be discussed next week. +This will be discussed next week. Before we develop our own codes for logistic regression, we end this lecture by studying the functionality that _Scikit-learn_ offers. !split -===== Cancer Data again now with Decision Trees and other Methods ===== +===== Wisconsin Cancer Data ===== !bc pycod import matplotlib.pyplot as plt import numpy as np