-
@@ -399,7 +399,7 @@ called ordered states (the spins tend to point in one direction, resulting in a net magnetic moment) while those above the critical temperature are called disordered. Since a finite lattice like this does not exhibit a clear sign of a phase transition we will mainly -stay with either orderer or disoredered phases. You could include the +stay with either orderer or disordered phases. You could include the critical phase if you want.
@@ -411,13 +411,24 @@ notes on slides here. + +
+To measure the performance of our network we evaluate how well it does +it data it has never seen before, i.e. the test data. We measure the +performance of the network using the accuracy score. The accuracy +is as you would expect just the number of images correctly labeled +divided by the total number of images. A perfect classifier will have +an accuracy score of \( 1 \). + +$$ +\text{Accuracy} = \frac{\sum_{i=1}^n I(t_i = y_i)}{n} , +$$ + +
+where \( I \) is the indicator function, \( 1 \) if \( t_i = y_i \) and \( 0 \) +otherwise, where \( t_i \) represents the target and \( y_i \) the outputs.
In order to find the optimal parameters of your logistic regressor you should @@ -428,8 +439,8 @@ Since we don't have so many data points, you may just code the standard gradient descent with a given learning rate, or even attempt to use the Newton-Raphson method. Alternatively, it may be useful for the next part on neural networks to implement a stochastic gradient -descent. For all gradient methods, you can use scikit-learn's toolbox for -optimization methods in order to test your own results. +descent with and without mini-batches. Stochastic gradient with mini-batches may give the best results. You could finally compare your code with the output from scikit-learn's toolbox for +optimization methods applied to logistic regression.
The notebook of Mehta et al is highly recommended in order to benchmark your code and results. @@ -449,10 +460,6 @@ to use the codes in the above lecture slides as starting points. Train your network and compare the results with those from your linear regression code. You can test your results against a similar code using _scikit_learn_ (see the examples in the above lecture notes) or tensorflow/keras. -
-You should have the same elements as in the regression examples, including the \( R2 \) score, the MSE, and bootstrap or cross-validation. -You may also need to think of using another activation function than the standard logistic function. -
A useful reference on the back progagation algorithm is Nielsen's book. It is an excellent read. diff --git a/doc/Projects/2018/Project2/html/Project2.html b/doc/Projects/2018/Project2/html/Project2.html index 7e8da5aaa..381cb78a5 100644 --- a/doc/Projects/2018/Project2/html/Project2.html +++ b/doc/Projects/2018/Project2/html/Project2.html @@ -120,7 +120,7 @@ MathJax.Hub.Config({
-
@@ -367,13 +367,24 @@ notes on slides here. + +
+To measure the performance of our network we evaluate how well it does +it data it has never seen before, i.e. the test data. We measure the +performance of the network using the accuracy score. The accuracy +is as you would expect just the number of images correctly labeled +divided by the total number of images. A perfect classifier will have +an accuracy score of \( 1 \). + +$$ +\text{Accuracy} = \frac{\sum_{i=1}^n I(t_i = y_i)}{n} , +$$ + +
+where \( I \) is the indicator function, \( 1 \) if \( t_i = y_i \) and \( 0 \) +otherwise, where \( t_i \) represents the target and \( y_i \) the outputs.
In order to find the optimal parameters of your logistic regressor you should @@ -384,8 +395,8 @@ Since we don't have so many data points, you may just code the standard gradient descent with a given learning rate, or even attempt to use the Newton-Raphson method. Alternatively, it may be useful for the next part on neural networks to implement a stochastic gradient -descent. For all gradient methods, you can use scikit-learn's toolbox for -optimization methods in order to test your own results. +descent with and without mini-batches. Stochastic gradient with mini-batches may give the best results. You could finally compare your code with the output from scikit-learn's toolbox for +optimization methods applied to logistic regression.
The notebook of Mehta et al is highly recommended in order to benchmark your code and results. @@ -405,10 +416,6 @@ to use the codes in the above lecture slides as starting points. Train your network and compare the results with those from your linear regression code. You can test your results against a similar code using _scikit_learn_ (see the examples in the above lecture notes) or tensorflow/keras. -
-You should have the same elements as in the regression examples, including the \( R2 \) score, the MSE, and bootstrap or cross-validation. -You may also need to think of using another activation function than the standard logistic function. -
A useful reference on the back progagation algorithm is Nielsen's book. It is an excellent read. diff --git a/doc/Projects/2018/Project2/ipynb/ipynb-Project2-src.tar.gz b/doc/Projects/2018/Project2/ipynb/ipynb-Project2-src.tar.gz index 23ce22ad2..70120d695 100644 Binary files a/doc/Projects/2018/Project2/ipynb/ipynb-Project2-src.tar.gz and b/doc/Projects/2018/Project2/ipynb/ipynb-Project2-src.tar.gz differ diff --git a/doc/Projects/2018/Project2/pdf/Project2.p.tex b/doc/Projects/2018/Project2/pdf/Project2.p.tex index 511dfff7e..2276120eb 100644 --- a/doc/Projects/2018/Project2/pdf/Project2.p.tex +++ b/doc/Projects/2018/Project2/pdf/Project2.p.tex @@ -155,7 +155,7 @@ Project 2 on Machine Learning, deadline November 5 % --- begin date --- \begin{center} -Oct 12, 2018 +Oct 16, 2018 \end{center} % --- end date --- @@ -371,7 +371,7 @@ called ordered states (the spins tend to point in one direction, resulting in a net magnetic moment) while those above the critical temperature are called disordered. Since a finite lattice like this does not exhibit a clear sign of a phase transition we will mainly -stay with either orderer or disoredered phases. You could include the +stay with either orderer or disordered phases. You could include the critical phase if you want. @@ -382,13 +382,23 @@ and write your own code for doing logistic regression, see the lecture notes on \href{{https://compphysics.github.io/MachineLearning/doc/pub/LogReg/html/LogReg-bs.html}}{logistic regression}. -You should include either bootstrap or cross-validation in setting up -your model (as done in project 1) and compute the $R2$ score and the -mean-square error. -We will use both ordered and disordered states to train -the logistic regressor and, once the supervised training procedure is -complete, you should evaluate the performance of your classification model on -unseen ordered, disordered and possibly critical states. + +In this case, to evaluate the model, we will use the so-called accuracy score +instead of the bootstrap or cross-validation as done in the standard linear regression part discussed in b). Examples of how to define the accuracy score can be found under the neural network slides, see for example the \href{{https://compphysics.github.io/MachineLearning/doc/pub/NeuralNet/html/._NeuralNet-bs047.html}}{slides here}. + +To measure the performance of our network we evaluate how well it does +it data it has never seen before, i.e.~the test data. We measure the +performance of the network using the \emph{accuracy} score. The accuracy +is as you would expect just the number of images correctly labeled +divided by the total number of images. A perfect classifier will have +an accuracy score of $1$. + +\[ +\text{Accuracy} = \frac{\sum_{i=1}^n I(t_i = y_i)}{n} , +\] + +where $I$ is the indicator function, $1$ if $t_i = y_i$ and $0$ +otherwise, where $t_i$ represents the target and $y_i$ the outputs. @@ -400,8 +410,8 @@ Since we don't have so many data points, you may just code the standard gradient descent with a given learning rate, or even attempt to use the Newton-Raphson method. Alternatively, it may be useful for the next part on neural networks to implement a stochastic gradient -descent. For all gradient methods, you can use \textbf{scikit-learn}'s toolbox for -optimization methods in order to test your own results. +descent with and without mini-batches. Stochastic gradient with mini-batches may give the best results. You could finally compare your code with the output from \textbf{scikit-learn}'s toolbox for +optimization methods applied to logistic regression. The notebook of \href{{https://physics.bu.edu/~pankajm/ML-Notebooks/HTML/NB_CVII-logreg_ising.html}}{Mehta et al} is highly recommended in order to benchmark your code and results. @@ -418,8 +428,6 @@ to use the codes in the above lecture slides as starting points. Train your network and compare the results with those from your linear regression code. You can test your results against a similar code using _scikit_learn_ (see the examples in the above lecture notes) or \textbf{tensorflow/keras}. -You should have the same elements as in the regression examples, including the $R2$ score, the MSE, and bootstrap or cross-validation. -You may also need to think of using another activation function than the standard logistic function. A useful reference on the back progagation algorithm is \href{{http://neuralnetworksanddeeplearning.com/}}{Nielsen's book}. It is an excellent read. @@ -568,6 +576,11 @@ of code developers and contributors keeps increasing. + + + + + % ------------------- end of main content --------------- % #ifdef PREAMBLE diff --git a/doc/Projects/2018/Project2/pdf/Project2.pdf b/doc/Projects/2018/Project2/pdf/Project2.pdf index 7e7fbf6cb..9362b25ec 100644 Binary files a/doc/Projects/2018/Project2/pdf/Project2.pdf and b/doc/Projects/2018/Project2/pdf/Project2.pdf differ diff --git a/doc/Projects/2018/Project2/pdf/Project2.tex b/doc/Projects/2018/Project2/pdf/Project2.tex index 1cf9e34d3..aeee4b93c 100644 --- a/doc/Projects/2018/Project2/pdf/Project2.tex +++ b/doc/Projects/2018/Project2/pdf/Project2.tex @@ -125,7 +125,7 @@ Project 2 on Machine Learning, deadline November 5 % --- begin date --- \begin{center} -Oct 12, 2018 +Oct 16, 2018 \end{center} % --- end date --- @@ -341,7 +341,7 @@ called ordered states (the spins tend to point in one direction, resulting in a net magnetic moment) while those above the critical temperature are called disordered. Since a finite lattice like this does not exhibit a clear sign of a phase transition we will mainly -stay with either orderer or disoredered phases. You could include the +stay with either orderer or disordered phases. You could include the critical phase if you want. @@ -352,13 +352,23 @@ and write your own code for doing logistic regression, see the lecture notes on \href{{https://compphysics.github.io/MachineLearning/doc/pub/LogReg/html/LogReg-bs.html}}{logistic regression}. -You should include either bootstrap or cross-validation in setting up -your model (as done in project 1) and compute the $R2$ score and the -mean-square error. -We will use both ordered and disordered states to train -the logistic regressor and, once the supervised training procedure is -complete, you should evaluate the performance of your classification model on -unseen ordered, disordered and possibly critical states. + +In this case, to evaluate the model, we will use the so-called accuracy score +instead of the bootstrap or cross-validation as done in the standard linear regression part discussed in b). Examples of how to define the accuracy score can be found under the neural network slides, see for example the \href{{https://compphysics.github.io/MachineLearning/doc/pub/NeuralNet/html/._NeuralNet-bs047.html}}{slides here}. + +To measure the performance of our network we evaluate how well it does +it data it has never seen before, i.e.~the test data. We measure the +performance of the network using the \emph{accuracy} score. The accuracy +is as you would expect just the number of images correctly labeled +divided by the total number of images. A perfect classifier will have +an accuracy score of $1$. + +\[ +\text{Accuracy} = \frac{\sum_{i=1}^n I(t_i = y_i)}{n} , +\] + +where $I$ is the indicator function, $1$ if $t_i = y_i$ and $0$ +otherwise, where $t_i$ represents the target and $y_i$ the outputs. @@ -370,8 +380,8 @@ Since we don't have so many data points, you may just code the standard gradient descent with a given learning rate, or even attempt to use the Newton-Raphson method. Alternatively, it may be useful for the next part on neural networks to implement a stochastic gradient -descent. For all gradient methods, you can use \textbf{scikit-learn}'s toolbox for -optimization methods in order to test your own results. +descent with and without mini-batches. Stochastic gradient with mini-batches may give the best results. You could finally compare your code with the output from \textbf{scikit-learn}'s toolbox for +optimization methods applied to logistic regression. The notebook of \href{{https://physics.bu.edu/~pankajm/ML-Notebooks/HTML/NB_CVII-logreg_ising.html}}{Mehta et al} is highly recommended in order to benchmark your code and results. @@ -388,8 +398,6 @@ to use the codes in the above lecture slides as starting points. Train your network and compare the results with those from your linear regression code. You can test your results against a similar code using _scikit_learn_ (see the examples in the above lecture notes) or \textbf{tensorflow/keras}. -You should have the same elements as in the regression examples, including the $R2$ score, the MSE, and bootstrap or cross-validation. -You may also need to think of using another activation function than the standard logistic function. A useful reference on the back progagation algorithm is \href{{http://neuralnetworksanddeeplearning.com/}}{Nielsen's book}. It is an excellent read. @@ -538,6 +546,11 @@ of code developers and contributors keeps increasing. + + + + + % ------------------- end of main content --------------- \end{document} diff --git a/doc/src/Projects/2018/Project2/Project2.do.txt b/doc/src/Projects/2018/Project2/Project2.do.txt index 621311326..23e6d47d6 100644 --- a/doc/src/Projects/2018/Project2/Project2.do.txt +++ b/doc/src/Projects/2018/Project2/Project2.do.txt @@ -225,7 +225,7 @@ called ordered states (the spins tend to point in one direction, resulting in a net magnetic moment) while those above the critical temperature are called disordered. Since a finite lattice like this does not exhibit a clear sign of a phase transition we will mainly -stay with either orderer or disoredered phases. You could include the +stay with either orderer or disordered phases. You could include the critical phase if you want. @@ -236,13 +236,25 @@ and write your own code for doing logistic regression, see the lecture notes on "logistic regression":"https://compphysics.github.io/MachineLearning/doc/pub/LogReg/html/LogReg-bs.html". -You should include either bootstrap or cross-validation in setting up -your model (as done in project 1) and compute the $R2$ score and the -mean-square error. -We will use both ordered and disordered states to train -the logistic regressor and, once the supervised training procedure is -complete, you should evaluate the performance of your classification model on -unseen ordered, disordered and possibly critical states. + +In this case, to evaluate the model, we will use the so-called accuracy score +instead of the bootstrap or cross-validation as done in the standard linear regression part discussed in b). Examples of how to define the accuracy score can be found under the neural network slides, see for example the "slides here":"https://compphysics.github.io/MachineLearning/doc/pub/NeuralNet/html/._NeuralNet-bs047.html". + +To measure the performance of our network we evaluate how well it does +it data it has never seen before, i.e. the test data. We measure the +performance of the network using the *accuracy* score. The accuracy +is as you would expect just the number of images correctly labeled +divided by the total number of images. A perfect classifier will have +an accuracy score of $1$. + +!bt +\[ +\text{Accuracy} = \frac{\sum_{i=1}^n I(t_i = y_i)}{n} , +\] +!et + +where $I$ is the indicator function, $1$ if $t_i = y_i$ and $0$ +otherwise, where $t_i$ represents the target and $y_i$ the outputs. @@ -254,8 +266,8 @@ Since we don't have so many data points, you may just code the standard gradient descent with a given learning rate, or even attempt to use the Newton-Raphson method. Alternatively, it may be useful for the next part on neural networks to implement a stochastic gradient -descent. For all gradient methods, you can use _scikit-learn_'s toolbox for -optimization methods in order to test your own results. +descent with and without mini-batches. Stochastic gradient with mini-batches may give the best results. You could finally compare your code with the output from _scikit-learn_'s toolbox for +optimization methods applied to logistic regression. The notebook of "Mehta et al":"https://physics.bu.edu/~pankajm/ML-Notebooks/HTML/NB_CVII-logreg_ising.html" is highly recommended in order to benchmark your code and results. @@ -273,8 +285,6 @@ to use the codes in the above lecture slides as starting points. Train your network and compare the results with those from your linear regression code. You can test your results against a similar code using _scikit_learn_ (see the examples in the above lecture notes) or _tensorflow/keras_. -You should have the same elements as in the regression examples, including the $R2$ score, the MSE, and bootstrap or cross-validation. -You may also need to think of using another activation function than the standard logistic function. A useful reference on the back progagation algorithm is "Nielsen's book":"http://neuralnetworksanddeeplearning.com/". It is an excellent read. @@ -400,3 +410,8 @@ of code developers and contributors keeps increasing. + + + + +