lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update on getting started

Browse Source
This commit is contained in:
mhjensen
2019-08-13 23:10:37 +02:00
parent 53136bbc0d
commit 73f987a013
420 changed files with 157728 additions and 5042 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md~
+122
View File
@@ -0,0 +1,122 @@
# FYS-STK3155/4155 Applied Data Analysis and Machine Learning, http://www.uio.no/studier/emner/matnat/fys/FYS-STK4155/index-eng.html
This site contains all material relevant for the course on Data Analysis and Machine Learning FYS-STK3155/4155.
## Course content
Probability theory and statistical methods play a central role in science. Nowadays we are
surrounded by huge amounts of data. For example, there are about one trillion web pages; more than one
hour of video is uploaded to YouTube every second, amounting to 10 years of content every
day; the genomes of 1000s of people, each of which has a length of more than a billion base pairs, have
been sequenced by various labs and so on.
This deluge of data calls for automated methods of data analysis,
which is exactly what machine
learning provides. In this course the approach is to define machine learning as a set of methods that can
automatically detect patterns in data, and then use the uncovered patterns to predict future
data, or to perform other kinds of decision making under uncertainty. Since many of these problems can be studied using
tools of probability theory, the aim of this course is to expose you to central methods in probability theory linked with machine learning.
This course covers thus topics like Monte Carlo methods and Markov chains, Bayesian statistics, error estimates, various regression methods, optimization of data and error analysis and central algorithms in machine learning.
The course has several numerical projects and numerical exercises that are meant to illustrate the theory.
## Learning outcomes
The course introduces a variety of central algorithms and methods
essential for studies of data analysis and machine learning. The course is project based and through the various projects, normally three, the students will be exposed to fundamental research problems in these fields, with the aim to reproduce state of the art scientific results. Both supervised and unsupervised methods will be covered. You will learn to develop and structure large codes for studying these systems, get acquainted with computing facilities and learn to handle large scientific projects. A good scientific and ethical conduct is emphasized throughout the course. More specifically, after this course you will
- Learn about basic data analysis, statistical analysis, Monte Carlo sampling, data optimization and machine learning;
- Be capable of extending the acquired knowledge to other systems and cases;
- Have an understanding of central algorithms used in data analysis and machine learning;
- Gain knowledge of central aspects of Monte Carlo methods, Markov chains, Gibbs samplers and their possible applications;
- Understand linear methods for regression and classification, from ordinary least squares, via Lasso and Ridge to Logistic regression;
- Learn about various neural networks and deep learning methods for supervised and unsupervised learning;
- Learn about about decision trees and random forests
- Learn about support vector machines and kernel transformations
- Reduction of data sets, from PCA to clustering, supervised and unsupervided methods
- Work on numerical projects to illustrate the theory. The projects play a central role and students are expected to know modern programming languages like Python or C++.
## Prerequisites
Basic knowledge in programming and mathematics, with an emphasis on linear algebra. Knowledge of Python or/and C++ as programming languages is required and experience with Jupiter notebook is recommended. Required courses are the equivalents to the University of Oslo mathematics courses MAT1100, MAT1110, MAT1120 and at least one of the corresponding computing and programming courses INF1000/INF1110 or MAT-INF1100/MAT-INF1100L/BIOS1100/KJM-INF1100.
## The course has two central parts
1. Statistical analysis and optimization of data
2. Machine learning
### Statistical analysis and optimization of data
The following topics will be covered
- Basic concepts, expectation values, variance, covariance, correlation functions and errors;
- Simpler models, binomial distribution, the Poisson distribution, simple and multivariate normal distributions;
- Central elements of Bayesian statistics and modeling;
- Central elements from linear algebra
- Gradient methods for data optimization
- Monte Carlo methods, Markov chains, Metropolis-Hastings algorithm;
- Linear methods for regression and classification;
- Estimation of errors using cross-validation, blocking, bootstrapping and jackknife methods;
- Practical optimization using Singular-value decomposition and least squares for parameterizing data.
### Machine learning, mainly supervised learning
The following topics will be covered
- Linear Regression and Logistic Regression;
- Neural networks and deep learning;
- Decisions trees and nearest neighbor algorithms
- Support vector machines
All the above topics will be supported by examples, hands-on exercises and project work.
Computational aspects play a central role and the students are
expected to work on numerical examples and projects which illustrate
the theory and methods. Some of the projects can be coordinated with the high-performance programming course IN4200.
## Practicalities
1. Four lectures per week, Fall semester, 10 ECTS;
2. Four hours of laboratory sessions for work on computational projects;
3. Three projects which are graded and count 1/3 each of the final grade;
4. A selected number of weekly assignments;
6. The course is part of the CS Master of Science program, but is open to other bachelor and Master of Science students at the University of Oslo;
7. Grading scale: Grades are awarded on a scale from A to F, where A is the best grade and F is a fail;
8. The course will be offered as a FYS-MAT4155 (Master of Science level) and a FYS-MAT3155 (senior undergraduate) course.
## Possible textbooks
_Recommended textbooks_:
- Trevor Hastie, Robert Tibshirani, Jerome H. Friedman, The Elements of Statistical Learning, Springer
- Aurelien Geron, Hands‑On Machine Learning with Scikit‑Learn and TensorFlow, O'Reilly
_General learning book on statistical analysis_:
- Christian Robert and George Casella, Monte Carlo Statistical Methods, Springer
- Peter Hoff, A first course in Bayesian statistical models, Springer
_General Machine Learning Books_:
- Kevin Murphy, Machine Learning: A Probabilistic Perspective, MIT Press
- Christopher M. Bishop, Pattern Recognition and Machine Learning, Springer
- David J.C. MacKay, Information Theory, Inference, and Learning Algorithms, Cambridge University Press
- David Barber, Bayesian Reasoning and Machine Learning, Cambridge University Press
## Links to relevant courses at the University of Oslo
The link here https://www.mn.uio.no/english/research/about/centre-focus/innovation/data-science/studies/ gives an excellent overview of courses on Machine learning at UiO.
- _STK2100 Machine learning and statistical methods for prediction and classification_ http://www.uio.no/studier/emner/matnat/math/STK2100/index-eng.html.
- _IN3050 Introduction to Artificial Intelligence and Machine Learning_ https://www.uio.no/studier/emner/matnat/ifi/IN3050/index-eng.html. Introductory course in machine learning and AI with an algorithmic approach.
- _STK-INF3000/4000 Selected Topics in Data Science_ http://www.uio.no/studier/emner/matnat/math/STK-INF3000/index-eng.html. The course provides insight into selected contemporary relevant topics within Data Science.
- _IN4080 Natural Language Processing_ https://www.uio.no/studier/emner/matnat/ifi/IN4080/index.html. Probabilistic and machine learning techniques applied to natural language processing.
- _STK-IN4300 Statistical learning methods in Data Science_ https://www.uio.no/studier/emner/matnat/math/STK-IN4300/index-eng.html. An advanced introduction to statistical and machine learning. For students with a good mathematics and statistics background.
- _INF4490 Biologically Inspired Computing_ http://www.uio.no/studier/emner/matnat/ifi/INF4490/. An introduction to self-adapting methods also called artificial intelligence or machine learning.
- _IN-STK5000 Adaptive Methods for Data-Based Decision Making_ https://www.uio.no/studier/emner/matnat/ifi/IN-STK5000/index-eng.html. Methods for adaptive collection and processing of data based on machine learning techniques.
- _IN5400/INF5860 Machine Learning for Image Analysis_ https://www.uio.no/studier/emner/matnat/ifi/IN5400/. An introduction to deep learning with particular emphasis on applications within Image analysis, but useful for other application areas too.
- _TEK5040 Deep learning for autonomous systems_ https://www.uio.no/studier/emner/matnat/its/TEK5040/. The course addresses advanced algorithms and architectures for deep learning with neural networks. The course provides an introduction to how deep-learning techniques can be used in the construction of key parts of advanced autonomous systems that exist in physical environments and cyber environments.
- _STK4051 Computational Statistics_ https://www.uio.no/studier/emner/matnat/math/STK4051/index-eng.html
- _STK4021 Applied Bayesian Analysis and Numerical Methods_ https://www.uio.no/studier/emner/matnat/math/STK4021/index-eng.html
doc/.DS_Store
Vendored
BIN
View File
Binary file not shown.
doc/DataFiles/.DS_Store
Vendored
BIN
View File
Binary file not shown.
doc/DataFiles/Finance/.DS_Store
Vendored
BIN
View File
Binary file not shown.
doc/LectureNotes/.book.copyright
+1
View File
@@ -0,0 +1 @@
{'holder': ['Morten Hjorth-Jensen'], 'year': '1999-2018', 'license': 'Released under CC Attribution-NonCommercial 4.0 license', 'cite doconce': False}
doc/LectureNotes/book.dlog
+231
View File
@@ -0,0 +1,231 @@
translating doconce text in book.do.txt to ipynb
*** error: could not open the file src/plot_Hudson.py used in
@@@CODE src/plot_Hudson.py
translating doconce text in book.do.txt to ipynb
copy complete file src/plot_Hudson.py (format: pypro)
copy complete file src/Hudson_Bay.py (format: pypro)
*** error: figure file "fig/Hudson_Bay_sim" does not exist!
translating doconce text in book.do.txt to ipynb
copy complete file src/plot_Hudson.py (format: pypro)
copy complete file src/Hudson_Bay.py (format: pypro)
figure file fig/Hudson_Bay_data:
can use fig/Hudson_Bay_data.png for format ipynb
figure file fig/Hudson_Bay_sim:
can use fig/Hudson_Bay_sim.png for format ipynb
collected all required additional files in ipynb-book-src.tar.gz which must be distributed with the notebook
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
output in book.ipynb
*** error: file has a mako construction ${\bf \hat{J}'
but seemingly no definition in <%...%>'
(it is not a command-line given mako variable either).
However, if this is a variable in a Makefile or Bash script
run with --no_mako - and you cannot use mako and Makefile or Bash variables
in the same document!
*** error: file has a mako construction ${\bf \hat{J}'
but seemingly no definition in <%...%>'
(it is not a command-line given mako variable either).
However, if this is a variable in a Makefile or Bash script
run with --no_mako - and you cannot use mako and Makefile or Bash variables
in the same document!
translating doconce text in book.do.txt to ipynb
*** error: found multiple labels:
eq:def_covariance eq:autocorrelation_time eq:error_estimate_corr_time
translating doconce text in book.do.txt to ipynb
copy complete file src/plot_Hudson.py (format: pypro)
copy complete file src/Hudson_Bay.py (format: pypro)
figure file fig/Hudson_Bay_data:
can use fig/Hudson_Bay_data.png for format ipynb
figure file fig/Hudson_Bay_sim:
can use fig/Hudson_Bay_sim.png for format ipynb
collected all required additional files in ipynb-book-src.tar.gz which must be distributed with the notebook
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
output in book.ipynb
translating doconce text in book.do.txt to ipynb
copy complete file src/plot_Hudson.py (format: pypro)
copy complete file src/Hudson_Bay.py (format: pypro)
figure file fig/Hudson_Bay_data:
can use fig/Hudson_Bay_data.png for format ipynb
figure file fig/Hudson_Bay_sim:
can use fig/Hudson_Bay_sim.png for format ipynb
collected all required additional files in ipynb-book-src.tar.gz which must be distributed with the notebook
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
*** warning: latex envir \begin{bmatrix} does not work well in Markdown.
Stick to \[ ... \], equation, equation*, align, or align*
environments in math environments.
output in book.ipynb
doc/LectureNotes/book.do.txt~
+12249
View File
File diff suppressed because it is too large Load Diff
doc/LectureNotes/src/Hudson_Bay.py~
+43
View File
@@ -0,0 +1,43 @@
import numpy as np
import matplotlib.pyplot as plt
def solver(m, H0, L0, dt, a, b, c, d, t0):
"""Solve the difference equations for H and L over m years
with time step dt (measured in years."""
num_intervals = int(m/float(dt))
t = np.linspace(t0, t0 + m, num_intervals+1)
H = np.zeros(t.size)
L = np.zeros(t.size)
print 'Init:', H0, L0, dt
H[0] = H0
L[0] = L0
for n in range(0, len(t)-1):
H[n+1] = H[n] + a*dt*H[n] - b*dt*H[n]*L[n]
L[n+1] = L[n] + d*dt*H[n]*L[n] - c*dt*L[n]
return H, L, t
# Load in data file
data = np.loadtxt('Hudson_Bay.csv', delimiter=',', skiprows=1)
# Make arrays containing x-axis and hares and lynx populations
t_e = data[:,0]
H_e = data[:,1]
L_e = data[:,2]
# Simulate using the model
H, L, t = solver(m=20, H0=34.91, L0=3.857, dt=0.1,
a=0.4807, b=0.02482, c=0.9272, d=0.02756,
t0=1900)
# Visualize simulations and data
plt.plot(t_e, H_e, 'b-+', t_e, L_e, 'r-o', t, H, 'm--', t, L, 'k--')
plt.xlabel('Year')
plt.ylabel('Numbers of hares and lynx')
plt.axis([1900, 1920, 0, 140])
plt.title(r'Population of hares and lynx 1900-1920 (x1000)')
plt.legend(('H_e', 'L_e', 'H', 'L'), loc='upper left')
plt.savefig('Hudson_Bay_sim.pdf')
plt.savefig('Hudson_Bay_sim.png')
plt.show()
doc/LectureNotes/src/plot_Hudson.py~
+19
View File
@@ -0,0 +1,19 @@
import numpy as np
from matplotlib import pyplot as plt
# Load in data file
data = np.loadtxt('src/Hudson_Bay.dat', delimiter=',', skiprows=1)
# Make arrays containing x-axis and hares and lynx populations
year = data[:,0]
hares = data[:,1]
lynx = data[:,2]
plt.plot(year, hares ,'b-+', year, lynx, 'r-o')
plt.axis([1900,1920,0, 100.0])
plt.xlabel(r'Year')
plt.ylabel(r'Numbers of hares and lynx ')
plt.legend(('Hares','Lynx'), loc='upper right')
plt.title(r'Population of hares and lynx from 1900-1920 (x1000)}')
plt.savefig('Hudson_Bay_data.pdf')
plt.savefig('Hudson_Bay_data.png')
plt.show()
doc/Programs/.DS_Store
Vendored
BIN
View File
Binary file not shown.
doc/Programs/ANN/cnnkeras.py~
+330
View File
@@ -0,0 +1,330 @@
# import necessary packages
import numpy as np
import matplotlib.pyplot as plt
from sklearn import datasets
# ensure the same random numbers appear every time
np.random.seed(0)
# display images in notebook
plt.rcParams['figure.figsize'] = (12,12)
# download MNIST dataset
digits = datasets.load_digits()
# define inputs and labels
inputs = digits.images
labels = digits.target
# RGB images have a depth of 3
# our images are grayscale so they should have a depth of 1
inputs = inputs[:,:,:,np.newaxis]
print("inputs = (n_inputs, pixel_width, pixel_height, depth) = " + str(inputs.shape))
print("labels = (n_inputs) = " + str(labels.shape))
# choose some random images to display
n_inputs = len(inputs)
indices = np.arange(n_inputs)
random_indices = np.random.choice(indices, size=5)
for i, image in enumerate(digits.images[random_indices]):
plt.subplot(1, 5, i+1)
plt.axis('off')
plt.imshow(image, cmap=plt.cm.gray_r, interpolation='nearest')
plt.title("Label: %d" % digits.target[random_indices[i]])
plt.show()
from keras.utils import to_categorical
from sklearn.model_selection import train_test_split
# representation of labels
labels = to_categorical(labels)
# split into train and test data
# one-liner from scikit-learn library
train_size = 0.8
test_size = 1 - train_size
X_train, X_test, Y_train, Y_test = train_test_split(inputs, labels, train_size=train_size,
test_size=test_size)
import tensorflow as tf
class ConvolutionalNeuralNetworkTensorflow:
def __init__(
self,
X_train,
Y_train,
X_test,
Y_test,
n_filters=10,
n_neurons_connected=50,
n_categories=10,
receptive_field=3,
stride=1,
padding=1,
epochs=10,
batch_size=100,
eta=0.1,
lmbd=0.0,
):
self.global_step = tf.Variable(0, dtype=tf.int32, trainable=False, name='global_step')
self.X_train = X_train
self.Y_train = Y_train
self.X_test = X_test
self.Y_test = Y_test
self.n_inputs, self.input_width, self.input_height, self.depth = X_train.shape
self.n_filters = n_filters
self.n_downsampled = int(self.input_width*self.input_height*n_filters / 4)
self.n_neurons_connected = n_neurons_connected
self.n_categories = n_categories
self.receptive_field = receptive_field
self.stride = stride
self.strides = [stride, stride, stride, stride]
self.padding = padding
self.epochs = epochs
self.batch_size = batch_size
self.iterations = self.n_inputs // self.batch_size
self.eta = eta
self.lmbd = lmbd
self.create_placeholders()
self.create_CNN()
self.create_loss()
self.create_optimiser()
self.create_accuracy()
def create_placeholders(self):
with tf.name_scope('data'):
self.X = tf.placeholder(tf.float32, shape=(None, self.input_width, self.input_height, self.depth), name='X_data')
self.Y = tf.placeholder(tf.float32, shape=(None, self.n_categories), name='Y_data')
def create_CNN(self):
with tf.name_scope('CNN'):
# Convolutional layer
self.W_conv = self.weight_variable([self.receptive_field, self.receptive_field, self.depth, self.n_filters], name='conv', dtype=tf.float32)
b_conv = self.weight_variable([self.n_filters], name='conv', dtype=tf.float32)
z_conv = tf.nn.conv2d(self.X, self.W_conv, self.strides, padding='SAME', name='conv') + b_conv
a_conv = tf.nn.relu(z_conv)
# 2x2 max pooling
a_pool = tf.nn.max_pool(a_conv, [1, 2, 2, 1], [1, 2, 2, 1], padding='SAME', name='pool')
# Fully connected layer
a_pool_flat = tf.reshape(a_pool, [-1, self.n_downsampled])
self.W_fc = self.weight_variable([self.n_downsampled, self.n_neurons_connected], name='fc', dtype=tf.float32)
b_fc = self.bias_variable([self.n_neurons_connected], name='fc', dtype=tf.float32)
a_fc = tf.nn.relu(tf.matmul(a_pool_flat, self.W_fc) + b_fc)
# Output layer
self.W_out = self.weight_variable([self.n_neurons_connected, self.n_categories], name='out', dtype=tf.float32)
b_out = self.bias_variable([self.n_categories], name='out', dtype=tf.float32)
self.z_out = tf.matmul(a_fc, self.W_out) + b_out
def create_loss(self):
with tf.name_scope('loss'):
softmax_loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits_v2(labels=self.Y, logits=self.z_out))
regularizer_loss_conv = tf.nn.l2_loss(self.W_conv)
regularizer_loss_fc = tf.nn.l2_loss(self.W_fc)
regularizer_loss_out = tf.nn.l2_loss(self.W_out)
regularizer_loss = self.lmbd*(regularizer_loss_conv + regularizer_loss_fc + regularizer_loss_out)
self.loss = softmax_loss + regularizer_loss
def create_accuracy(self):
with tf.name_scope('accuracy'):
probabilities = tf.nn.softmax(self.z_out)
predictions = tf.argmax(probabilities, 1)
labels = tf.argmax(self.Y, 1)
correct_predictions = tf.equal(predictions, labels)
correct_predictions = tf.cast(correct_predictions, tf.float32)
self.accuracy = tf.reduce_mean(correct_predictions)
def create_optimiser(self):
with tf.name_scope('optimizer'):
self.optimizer = tf.train.GradientDescentOptimizer(learning_rate=self.eta).minimize(self.loss, global_step=self.global_step)
def weight_variable(self, shape, name='', dtype=tf.float32):
initial = tf.truncated_normal(shape, stddev=0.1)
return tf.Variable(initial, name=name, dtype=dtype)
def bias_variable(self, shape, name='', dtype=tf.float32):
initial = tf.constant(0.1, shape=shape)
return tf.Variable(initial, name=name, dtype=dtype)
def fit(self):
data_indices = np.arange(self.n_inputs)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
for i in range(self.epochs):
for j in range(self.iterations):
chosen_datapoints = np.random.choice(data_indices, size=self.batch_size, replace=False)
batch_X, batch_Y = self.X_train[chosen_datapoints], self.Y_train[chosen_datapoints]
sess.run([CNN.loss, CNN.optimizer],
feed_dict={CNN.X: batch_X,
CNN.Y: batch_Y})
accuracy = sess.run(CNN.accuracy,
feed_dict={CNN.X: batch_X,
CNN.Y: batch_Y})
step = sess.run(CNN.global_step)
self.train_loss, self.train_accuracy = sess.run([CNN.loss, CNN.accuracy],
feed_dict={CNN.X: self.X_train,
CNN.Y: self.Y_train})
self.test_loss, self.test_accuracy = sess.run([CNN.loss, CNN.accuracy],
feed_dict={CNN.X: self.X_test,
CNN.Y: self.Y_test})
epochs = 100
batch_size = 100
n_filters = 10
n_neurons_connected = 50
n_categories = 10
eta_vals = np.logspace(-5, 1, 7)
lmbd_vals = np.logspace(-5, 1, 7)
CNN_tf = np.zeros((len(eta_vals), len(lmbd_vals)), dtype=object)
for i, eta in enumerate(eta_vals):
for j, lmbd in enumerate(lmbd_vals):
CNN = ConvolutionalNeuralNetworkTensorflow(X_train, Y_train, X_test, Y_test,
n_filters=n_filters, n_neurons_connected=n_neurons_connected,
n_categories=n_categories, epochs=epochs, batch_size=batch_size,
eta=eta, lmbd=lmbd)
CNN.fit()
print("Learning rate = ", eta)
print("Lambda = ", lmbd)
print("Test accuracy: %.3f" % CNN.test_accuracy)
print()
CNN_tf[i][j] = CNN
# visual representation of grid search
# uses seaborn heatmap, could probably do this in matplotlib
import seaborn as sns
sns.set()
train_accuracy = np.zeros((len(eta_vals), len(lmbd_vals)))
test_accuracy = np.zeros((len(eta_vals), len(lmbd_vals)))
for i in range(len(eta_vals)):
for j in range(len(lmbd_vals)):
CNN = CNN_tf[i][j]
train_accuracy[i][j] = CNN.train_accuracy
test_accuracy[i][j] = CNN.test_accuracy
fig, ax = plt.subplots(figsize = (10, 10))
sns.heatmap(train_accuracy, annot=True, ax=ax, cmap="viridis")
ax.set_title("Training Accuracy")
ax.set_ylabel("$\eta$")
ax.set_xlabel("$\lambda$")
plt.show()
fig, ax = plt.subplots(figsize = (10, 10))
sns.heatmap(test_accuracy, annot=True, ax=ax, cmap="viridis")
ax.set_title("Test Accuracy")
ax.set_ylabel("$\eta$")
ax.set_xlabel("$\lambda$")
plt.show()
from keras.models import Sequential
from keras.layers.convolutional import Conv2D
from keras.layers.convolutional import MaxPooling2D
from keras.layers import Flatten
from keras.layers import Dense
from keras.regularizers import l2
from keras.optimizers import SGD
def create_convolutional_neural_network_keras(input_shape, receptive_field,
n_filters, n_neurons_connected, n_categories,
eta, lmbd):
model = Sequential()
model.add(Conv2D(n_filters, (receptive_field, receptive_field), input_shape=input_shape, padding='same',
activation='relu', kernel_regularizer=l2(lmbd)))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(n_neurons_connected, activation='relu', kernel_regularizer=l2(lmbd)))
model.add(Dense(n_categories, activation='softmax', kernel_regularizer=l2(lmbd)))
sgd = SGD(lr=eta)
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])
return model
epochs = 100
batch_size = 100
input_shape = X_train.shape[1:4]
receptive_field = 3
n_filters = 10
n_neurons_connected = 50
n_categories = 10
eta_vals = np.logspace(-5, 1, 7)
lmbd_vals = np.logspace(-5, 1, 7)
CNN_keras = np.zeros((len(eta_vals), len(lmbd_vals)), dtype=object)
for i, eta in enumerate(eta_vals):
for j, lmbd in enumerate(lmbd_vals):
CNN = create_convolutional_neural_network_keras(input_shape, receptive_field,
n_filters, n_neurons_connected, n_categories,
eta, lmbd)
CNN.fit(X_train, Y_train, epochs=epochs, batch_size=batch_size, verbose=0)
scores = CNN.evaluate(X_test, Y_test)
CNN_keras[i][j] = CNN
print("Learning rate = ", eta)
print("Lambda = ", lmbd)
print("Test accuracy: %.3f" % scores[1])
print()
# visual representation of grid search
# uses seaborn heatmap, could probably do this in matplotlib
import seaborn as sns
sns.set()
train_accuracy = np.zeros((len(eta_vals), len(lmbd_vals)))
test_accuracy = np.zeros((len(eta_vals), len(lmbd_vals)))
for i in range(len(eta_vals)):
for j in range(len(lmbd_vals)):
CNN = CNN_keras[i][j]
train_accuracy[i][j] = CNN.evaluate(X_train, Y_train)[1]
test_accuracy[i][j] = CNN.evaluate(X_test, Y_test)[1]
fig, ax = plt.subplots(figsize = (10, 10))
sns.heatmap(train_accuracy, annot=True, ax=ax, cmap="viridis")
ax.set_title("Training Accuracy")
ax.set_ylabel("$\eta$")
ax.set_xlabel("$\lambda$")
plt.show()
fig, ax = plt.subplots(figsize = (10, 10))
sns.heatmap(test_accuracy, annot=True, ax=ax, cmap="viridis")
ax.set_title("Test Accuracy")
ax.set_ylabel("$\eta$")
ax.set_xlabel("$\lambda$")
plt.show()
doc/Programs/DimRed/covariance.py~
+34
View File
@@ -0,0 +1,34 @@
#The covariance matrix and its eigenvalues the hard way
from random import random, seed
import numpy as np
def covariance(x, y, n):
sum = 0.0
mean_x = np.mean(x)
mean_y = np.mean(y)
for i in range(0, n):
sum += (x[(i)]-mean_x)*(y[i]-mean_y)
return sum/n
n = 10
x = np.random.normal(size=n)
y = np.random.normal(size=n)
z = x*x*x+y*y 0.5*np.random.normal(size=n)
covxx = covariance(x,x,n)
covxy = covariance(x,y,n)
covxz = covariance(x,z,n)
covyy = covariance(y,y,n)
covyz = covariance(y,z,n)
covzz = covariance(z,z,n)
SigmaCov = np.array([ [covxx, covxy, covxz], [covxy, covyy, covyz], [covxz, covyz, covzz]])
print(SigmaCov)
EigValues, EigVectors = np.linalg.eig(SigmaCov)
# sort eigenvectors and eigenvalues
permute = EigValues.argsort()
EigValues = EigValues[permute]
EigVectors = EigVectors[:,permute]
print(EigValues)
print(EigVectors)
doc/Programs/Finance/agents.py~
+38
View File
@@ -0,0 +1,38 @@
# Simulation of financial transations with or without saving/taxation on transaction
# If lambda =0.0, no saving/taxation
# See Patriarca et al http://www.sciencedirect.com/science/article/pii/S0378437104004327
#!/usr/bin/env python
import numpy as np
import matplotlib.mlab as mlab
import matplotlib.pyplot as plt
import random
# initialize the rng with a seed
random.seed()
# Hard coding of input parameters
Agents = 500
MCcounts = 1000
Transactions = 100000
startMoney = 1.0
Lambda = 0.5
FinancialAgents = startMoney*np.ones(Agents)
for i in range (1, MCcounts, 1):
for j in range (1, Transactions, 1):
agent_i = int(Agents*random.random())
agent_j = int(Agents*random.random())
epsilon = random.random()
if agent_i != agent_j:
m1 = Lambda*FinancialAgents[agent_i] + (1-Lambda)*epsilon*(FinancialAgents[agent_i] + FinancialAgents[agent_j])
m2 = Lambda*FinancialAgents[agent_j] + (1-Lambda)*(1-epsilon)*(FinancialAgents[agent_i] + FinancialAgents[agent_j])
FinancialAgents[agent_i] = m1
FinancialAgents[agent_j] = m2
# the histogram of the data
n, bins, patches = plt.hist(FinancialAgents, 50, facecolor='green')
plt.xlabel('$x$')
plt.ylabel('Distribution of wealth')
plt.title(r'Money')
plt.axis([0, 10, 0, 100])
plt.grid(True)
plt.show()
doc/Programs/RandomWalks/OneDimParticle.py~
+45
View File
@@ -0,0 +1,45 @@
# Program to test the Metropolis algorithm with one particle at given temp in
# one dimension
#!/usr/bin/env python
import numpy as np
import matplotlib.mlab as mlab
import matplotlib.pyplot as plt
import random
from math import sqrt, exp, log
# initialize the rng with a seed
random.seed()
# Hard coding of input parameters
MCcycles = 100000
Temperature = 2.0
beta = 1./Temperature
InitialVelocity = -2.0
CurrentVelocity = InitialVelocity
Energy = 0.5*InitialVelocity*InitialVelocity
VelocityRange = 10*sqrt(Temperature)
VelocityStep = 2*VelocityRange/10.
AverageEnergy = Energy
AverageEnergy2 = Energy*Energy
VelocityValues = np.zeros(MCcycles)
# The Monte Carlo sampling with Metropolis starts here
for i in range (1, MCcycles, 1):
TrialVelocity = CurrentVelocity + (2.0*random.random() - 1.0)*VelocityStep
EnergyChange = 0.5*(TrialVelocity*TrialVelocity -CurrentVelocity*CurrentVelocity);
if random.random() <= exp(-beta*EnergyChange):
CurrentVelocity = TrialVelocity
Energy += EnergyChange
VelocityValues[i] = CurrentVelocity
AverageEnergy += Energy
AverageEnergy2 += Energy*Energy
#Final averages
AverageEnergy = AverageEnergy/MCcycles
AverageEnergy2 = AverageEnergy2/MCcycles
Variance = AverageEnergy2 - AverageEnergy*AverageEnergy
print(AverageEnergy, Variance)
n, bins, patches = plt.hist(VelocityValues, 400, facecolor='green')
plt.xlabel('$v$')
plt.ylabel('Velocity distribution P(v)')
plt.title(r'Velocity histogram at $k_BT=2$')
plt.axis([-5, 5, 0, 600])
plt.grid(True)
plt.show()
doc/Programs/ResamplingAnalysisScripts/README.md~
+19
View File
@@ -0,0 +1,19 @@
# ResamplingAnalysisScripts
## Sample Scripts for data Analysis
So far this is a simple python script (should be made parallel...) to perform resampling of a data set. Methods used are __Bootstrapping__, __Jackknife__ and __Blocking__.
## Usage
Simply run `python analysis.py FILENAME.xxx [NLINES]`
Where `FILENAME` is expected to have a 3 charachter extension `NLINES` (optional) is the number of lines in the file to read and process (default is the whole file, but it gets very slow above 2-3 hundred thousand entries)
Ouput is located into the `FILENAME/` folder.
If more than 10⁵ lines are specified the autocorrelation function won't be computed, as it would take too long.
The `gaussian.dat` dataset has been generated with numpy, as a proof of concept. It represents a normally distributed set of 5x10⁵ elements with `std = 0.05`. One will notice that the estimate on the error of the central value is greatly improved by all resampling methods.
`energy.dat` is an autocorrelated data set, with autocorrelation time of roughly 200. It is useful to see the use of blocking on this dataset as a convenient method to estimate the autocorrelation time (compare the elapsed time on the different methods).
In the `plaquette.dat` file there is a small data set (just 1000 samples) and it shows the strenght of using resampling methods to better estimate the error on the central value as opposed to the standard deviation.
doc/Programs/SVD/Fortran/simplefit.dat~
Executable
+9
View File
@@ -0,0 +1,9 @@
8 2
0.001 -2.89017 0.00073621
0.002 -2.88946 0.00052732
0.005 -2.89067 0.00055038
0.010 -2.89091 0.00040973
0.015 -2.89084 0.00034278
0.02 -2.89086 0.00029315
0.025 -2.89059 0.00034278
0.03 -2.89077 0.00025017
doc/Programs/Sampling/.DS_Store
Vendored
BIN
View File
Binary file not shown.
doc/Projects/2018/Project1/pdf/.DS_Store
Vendored
BIN
View File
Binary file not shown.
doc/Projects/2018/Project2/html/._Project2-bs000.html
+611
View File
@@ -0,0 +1,611 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="description" content="Project 2 on Machine Learning, deadline November 12">
<title>Project 2 on Machine Learning, deadline November 12</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Classification and Regression, from linear and logistic '
'regression to neural networks',
2,
None,
'___sec0'),
('Part a): Producing the data for the one-dimensional Ising '
'model',
3,
None,
'___sec1'),
('Part b): Estimating the coupling constant of the '
'one-dimensional Ising model using linear regression',
3,
None,
'___sec2'),
('Part c): Determine the phase of the two-dimensional Ising '
'model',
3,
None,
'___sec3'),
('Part d): Regression analysis of the one-dimensional Ising '
'model using neural networks',
3,
None,
'___sec4'),
('Part e): Classifying the Ising model phase using neural '
'networks',
3,
None,
'___sec5'),
('Part f) Critical evaluation of the various algorithms',
3,
None,
'___sec6'),
('Background literature', 2, None, '___sec7'),
('Introduction to numerical projects', 2, None, '___sec8'),
('Format for electronic delivery of report and programs',
2,
None,
'___sec9'),
('Software and needed installations', 2, None, '___sec10')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Project2-bs.html">Project 2 on Machine Learning, deadline November 12</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="#___sec0" style="font-size: 80%;"><b>Classification and Regression, from linear and logistic regression to neural networks</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec1" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Part a): Producing the data for the one-dimensional Ising model</a></li>
<!-- navigation toc: --> <li><a href="#___sec2" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Part b): Estimating the coupling constant of the one-dimensional Ising model using linear regression</a></li>
<!-- navigation toc: --> <li><a href="#___sec3" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Part c): Determine the phase of the two-dimensional Ising model</a></li>
<!-- navigation toc: --> <li><a href="#___sec4" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Part d): Regression analysis of the one-dimensional Ising model using neural networks</a></li>
<!-- navigation toc: --> <li><a href="#___sec5" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Part e): Classifying the Ising model phase using neural networks</a></li>
<!-- navigation toc: --> <li><a href="#___sec6" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Part f) Critical evaluation of the various algorithms</a></li>
<!-- navigation toc: --> <li><a href="#___sec7" style="font-size: 80%;"><b>Background literature</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec8" style="font-size: 80%;"><b>Introduction to numerical projects</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec9" style="font-size: 80%;"><b>Format for electronic delivery of report and programs</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec10" style="font-size: 80%;"><b>Software and needed installations</b></a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0000"></a>
<!-- ------------------- main content ---------------------- -->
<div class="jumbotron">
<center><h1>Project 2 on Machine Learning, deadline November 12</h1></center> <!-- document title -->
<p>
<!-- author(s): <a href="http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html" target="_self">Data Analysis and Machine Learning FYS-STK3155/FYS4155</a> -->
<center>
<b><a href="http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html" target="_self">Data Analysis and Machine Learning FYS-STK3155/FYS4155</a></b>
</center>
<p>
<!-- institution -->
<center><b>Department of Physics, University of Oslo, Norway</b></center>
<br>
<p>
<center><h4>Nov 2, 2018</h4></center> <!-- date -->
<br>
<p>
</div> <!-- end jumbotron -->
<h2 id="___sec0" class="anchor">Classification and Regression, from linear and logistic regression to neural networks </h2>
<p>
The main aim of this project is to study both classification and
regression problems, starting with the regression algorithms studied
in project 1. We will include logistic regresion for classification
problems and write our own multilayer perceptron code for studying
both regression and classification problems. The codes developed in
project 1, including bootstrap and/or cross-validation as well as the
computation of the mean-squared error and the R2 score function can
also be utilized (and included in logistic regression and the neural
network codes) in the present analysis.
<p>
We will use the so-called Ising model for our training data and will
focus on supervised training. We will follow closely the recent
article of <a href="https://arxiv.org/abs/1803.08823" target="_self">Mehta et al, arXiv
1803.08823</a>. This article stands
out as an excellent review on machine learning (ML) algorithms.
The added benefit is that each figure and
model presented in <a href="https://physics.bu.edu/~pankajm/MLnotebooks.html" target="_self">this article is accompanied by its jupyter
notebook</a>. This
means that we can start using these and compare with our own results.
They provide also the data set for the regression and classification
analysis that we will explore. In this sense, with their available
notebooks, it makes life easier since we can compare our own codes
with their codes.
<p>
With the abovementioned configurations we will determine, using first
various regression methods, the value of the coupling constant for the
energy of the one-dimensional Ising model. Thereafter, we will use the
two-dimensional data, but now computed at different temperatures, in
order to classify the phase of the Ising model. Below the critical
temperature, the system will be in a so-called ferromagnetic
phase. Close to the critical temperature, the final magnetization
becomes smaller and smaller in absolute value while above the critical
temperature, the net magnetization is zero. This classification case,
that is the two-dimensional Ising model, will be studied using
logistic regression and deep neural networks. The aim is to develop
your own logistic regression code for the classification of the phases
(this is a binary model) and your multilayer perceptron code for the
classification and regression case. You can compare your own results with those obtained
using <b>scikit-learn</b> or <b>tensorflow</b> or other Python packages such as <b>keras</b> or other.
<p>
Feel free to use the notebooks to benchmark your code. If you wish to
write your own C++ or Fortran program for say a multilayer neural network
model and a logistic regression model, please feel free to do so.
<h3 id="___sec1" class="anchor">Part a): Producing the data for the one-dimensional Ising model </h3>
<p>
The model we will employ in our studies is the so-called <a href="https://en.wikipedia.org/wiki/Ising_model" target="_self">Ising
model</a>. Together with
models like the <a href="https://en.wikipedia.org/wiki/Potts_model" target="_self">Potts
model</a> and similar
so-called lattice models, the Ising model has been widely studied in
mathematics (in statistics in particular), physics, <a href="https://journals.aps.org/pre/abstract/10.1103/PhysRevE.93.062402" target="_self">life
science</a>,
chemistry and even in the <a href="https://www.springer.com/gp/book/9781461420316" target="_self">social sciences in order to model social
behavior</a>. It is a
simple binary value system where the variables of the model (spins often in
physics) can take two values only, for example \( \pm 1 \) or \( 0 \) and \( 1 \).
The system exhibits a phase transition in two or higher dimensions and
the first person to find the analytical expressions for various
expectation values was the Norwegian chemist <a href="https://en.wikipedia.org/wiki/Lars_Onsager" target="_self">Lars
Onsager</a> (Nobel prize in
chemistry) after a tour de force mathematics exercise.
<p>
In our discussions here we will stay with a physicist's approach and
call the variables for spin. You could replace this with any other
type of binary variables, ranging from a two political parties to blue
and red spheres. In its simplest form we define the energy of the
system as
$$
\begin{equation*}
E=-J\sum_{< kl>}^{N}s_ks_l,
\end{equation*}
$$
with \( s_k=\pm 1 \), \( N \) is the total number of spins,
\( J \) is a coupling constant expressing the strength of the interaction
between neighboring spins.
<p>
The symbol \( < kl> \) indicates that we sum over nearest
neighbors only.
Notice that for \( J>0 \) it is energetically favorable for neighboring spins
to be aligned. This feature leads to, at low enough temperatures,
a cooperative phenomenon called spontaneous magnetization. That is,
through interactions between nearest neighbors, a given magnetic
moment can influence the alignment of spins that are separated
from the given spin by a macroscopic distance. These long range correlations
between spins are associated with a long-range order in which
the lattice has a net magnetization in the absence of a magnetic field.
<p>
We start by considering the one-dimensional Ising model with nearest neighbor interactions. This model does not exhibit any phase transition.
<p>
Consider the 1D Ising model with nearest-neighbor interactions
$$
\begin{equation*}
E[\hat{s}]=-J\sum_{j=1}^{N}s_{j}s_{j+1},
\end{equation*}
$$
<p>
on a chain of length \( N \) with so-called periodic boundary conditions and \( S_j=\pm 1 \) Ising spin variables.
In one dimension, this model has no phase transition at finite temperature.
<p>
In the Python code below we generate, with a coupling coefficient set to \( J=1 \), a large number of spin configurations say \( 10000 \) as shown in the code below.
It means that our data will be a set of \( i=1\ldots n \) points of the form
\( \{(E[\boldsymbol{s}^i],\boldsymbol{s}^i)\} \).
Our task is to find the value of \( J \) from the data set using linear regression.
<p>
Here is the Python code you need to generate the training data, see
also the <a href="https://physics.bu.edu/~pankajm/ML-Notebooks/HTML/NB_CVI-linreg_ising.html" target="_self">notebook of Mehta et
al</a>.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">scipy.sparse</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">sp</span>
np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>seed(<span style="color: #666666">12</span>)
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">warnings</span>
<span style="color: #408080; font-style: italic">#Comment this to turn on warnings</span>
warnings<span style="color: #666666">.</span>filterwarnings(<span style="color: #BA2121">&#39;ignore&#39;</span>)
<span style="color: #408080; font-style: italic">### define Ising model aprams</span>
<span style="color: #408080; font-style: italic"># system size</span>
L<span style="color: #666666">=40</span>
<span style="color: #408080; font-style: italic"># create 10000 random Ising states</span>
states<span style="color: #666666">=</span>np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>choice([<span style="color: #666666">-1</span>, <span style="color: #666666">1</span>], size<span style="color: #666666">=</span>(<span style="color: #666666">10000</span>,L))
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">ising_energies</span>(states,L):
<span style="color: #BA2121; font-style: italic">&quot;&quot;&quot;</span>
<span style="color: #BA2121; font-style: italic"> This function calculates the energies of the states in the nn Ising Hamiltonian</span>
<span style="color: #BA2121; font-style: italic"> &quot;&quot;&quot;</span>
J<span style="color: #666666">=</span>np<span style="color: #666666">.</span>zeros((L,L),)
<span style="color: #008000; font-weight: bold">for</span> i <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span>(L):
J[i,(i<span style="color: #666666">+1</span>)<span style="color: #666666">%</span>L]<span style="color: #666666">-=1.0</span>
<span style="color: #408080; font-style: italic"># compute energies</span>
E <span style="color: #666666">=</span> np<span style="color: #666666">.</span>einsum(<span style="color: #BA2121">&#39;...i,ij,...j-&gt;...&#39;</span>,states,J,states)
<span style="color: #008000; font-weight: bold">return</span> E
<span style="color: #408080; font-style: italic"># calculate Ising energies</span>
energies<span style="color: #666666">=</span>ising_energies(states,L)
</pre></div>
<p>
We can now recast the problem as a linear regression model using our codes from project 1.
The way we are going to build our model mimicks the way we could think of finding say the gravitional constant for the graviational force between two planets.
In the absence of any prior knowledge, one sensible choice is the all-to-all Ising model
$$
E_\mathrm{model}[\boldsymbol{s}^i] = - \sum_{j=1}^N \sum_{k=1}^N J_{j,k}s_{j}^is_{k}^i.
$$
<p>
Here \( i \) represents a particular spin configuration (one of the possible \( n \) configurations we generated with the code above).
<p>
This model is uniquely defined by the non-local coupling strengths \( J_{jk} \) which we want to learn.
The model is linear in \( \mathbf{J} \) which makes it possible to use linear regression.
<p>
To apply linear regression, we recast this model in the form
$$
E_\mathrm{model}^i \equiv \mathbf{X}^i \cdot \mathbf{J},
$$
<p>
where the vectors \( \mathbf{X}^i \) represent all two-body interactions
\( \{s_{j}^is_{k}^i \}_{j,k=1}^N \), and the index \( i \) runs over the
samples in the data set. To make the analogy complete, we can also
represent the dot product by a single index \( p = \{j,k\} \),
i.e. \( \mathbf{X}^i \cdot \mathbf{J}=X^i_pJ_p \). Note that the
regression model does not include the minus sign, so we expect to
learn negative \( J \)'s.
<p>
With these preliminaries, we are now ready to reutilize our codes from project 1.
<h3 id="___sec2" class="anchor">Part b): Estimating the coupling constant of the one-dimensional Ising model using linear regression </h3>
<p>
We start with the one-dimensional Ising model and use the data we have
generated with \( J=1 \) in the previous point. Use linear regression,
Lasso and Ridge regression as done in project 1. You can compare your
results with those of <a href="https://physics.bu.edu/~pankajm/ML-Notebooks/HTML/NB_CVI-linreg_ising.html" target="_self">Mehta
et al.</a>.
Make sure it is the 1D data which is used.
<p>
Discuss the methods and how they perform in computing the coupling
constant \( J \) and include a bias-variance analysis using either
cross-validation or bootstrap. Discuss also the mean squared error and
the \( R2 \) score as measures to assess your model.
<p>
Give a critical analysis of your results.
<h3 id="___sec3" class="anchor">Part c): Determine the phase of the two-dimensional Ising model </h3>
<p>
We switch now to binary classification methods and use logistic
regression to define the phases of the Ising model. This means that we switch to the two-dimensional Ising model
and use the data sets generated by <a href="https://physics.bu.edu/~pankajm/ML-Review-Datasets/isingMC/" target="_self">Mehta et al</a>
These energies and their corresponding spin orientation configurations
represent then your data. We will use a fixed lattice of \( L\times L =
40 \times 40 \) spins in two dimensions. The link above contains data for several temperatures.
The theoretical critical temperature for a phase transition is \( T_C\approx 2.269 \) in units of energy.
However, for a finite lattice the results representing the critical temperature are slightly higher (\( T_C \approx 2.3 \)).
<p>
Our goal here, using logistic regression, is to train our model to
predict the phase of a sample given the spin configuration, whether it
represents a state above the critical temperature or below. The
configurations representing states below the critical temperature are
called ordered states (the spins tend to point in one direction,
resulting in a net magnetic moment) while those above the critical
temperature are called disordered. Since a finite lattice like this
does not exhibit a clear sign of a phase transition we will mainly
stay with either orderer or disordered phases. You could include the
critical phase if you want.
<p>
Your aim here is thus to read in these data (use the examples from
<a href="https://physics.bu.edu/~pankajm/ML-Notebooks/HTML/NB_CVII-logreg_ising.html" target="_self">Mehta et
al</a>)
and write your own code for doing logistic regression, see the lecture
notes on <a href="https://compphysics.github.io/MachineLearning/doc/pub/LogReg/html/LogReg-bs.html" target="_self">logistic
regression</a>.
<p>
In this case, to evaluate the model, we will use the so-called accuracy score
instead of the bootstrap or cross-validation as done in the standard linear regression part discussed in b). Examples of how to define the accuracy score can be found under the neural network slides, see for example the <a href="https://compphysics.github.io/MachineLearning/doc/pub/NeuralNet/html/._NeuralNet-bs047.html" target="_self">slides here</a>.
<p>
To measure the performance of our network we evaluate how well it does
it data it has never seen before, i.e. the test data. We measure the
performance of the network using the <em>accuracy</em> score. The accuracy
is as you would expect just the number of images correctly labeled
divided by the total number of images. A perfect classifier will have
an accuracy score of \( 1 \).
$$
\text{Accuracy} = \frac{\sum_{i=1}^n I(t_i = y_i)}{n} ,
$$
<p>
where \( I \) is the indicator function, \( 1 \) if \( t_i = y_i \) and \( 0 \)
otherwise, where \( t_i \) represents the target and \( y_i \) the outputs.
<p>
In order to find the optimal parameters of your logistic regressor you should
include a gradient descent solver, as discussed in the <a href="https://compphysics.github.io/MachineLearning/doc/pub/Splines/html/Splines-bs.html" target="_self">gradient
descent
lectures</a>.
Since we don't have so many data points, you may just code the
standard gradient descent with a given learning rate, or even attempt
to use the Newton-Raphson method. Alternatively, it may be useful for
the next part on neural networks to implement a stochastic gradient
descent with and without mini-batches. Stochastic gradient with mini-batches may give the best results. You could finally compare your code with the output from <b>scikit-learn</b>'s toolbox for
optimization methods applied to logistic regression.
<p>
The notebook of <a href="https://physics.bu.edu/~pankajm/ML-Notebooks/HTML/NB_CVII-logreg_ising.html" target="_self">Mehta et al</a> is highly recommended in order to benchmark your code and results.
<h3 id="___sec4" class="anchor">Part d): Regression analysis of the one-dimensional Ising model using neural networks </h3>
<p>
Your aim now, and this is the central part of this project, is to
write to your own multilayer perceptron model implementing the back
propagation algorithm discussed in the <a href="https://compphysics.github.io/MachineLearning/doc/pub/NeuralNet/html/NeuralNet-bs.html" target="_self">lecture
slides</a>. We
start with the regression case discussed in parts a) and b) but train
now the network to find the optimal weights and biases. You are free
to use the codes in the above lecture slides as starting points.
<p>
Train your network and compare the results with those from your linear regression code.
You can test your results against a similar code using _scikit_learn_ (see the examples in the above lecture notes) or <b>tensorflow/keras</b>.
<p>
A useful reference on the back progagation algorithm is <a href="http://neuralnetworksanddeeplearning.com/" target="_self">Nielsen's book</a>. It is an excellent read.
<h3 id="___sec5" class="anchor">Part e): Classifying the Ising model phase using neural networks </h3>
<p>
Finally, change now your cost function to the \( log \) cross-entropy classification cost function for the case discussed in part c). Train your network again and
compare the results with those from your logistic regression code i c).
Here again you can compare your results with those of <a href="https://physics.bu.edu/~pankajm/ML-Notebooks/HTML/NB_CIX-DNN_ising_TFlow.html" target="_self">Mehta et al</a>. There they used <b>tensorflow</b> to classify the phases.
<h3 id="___sec6" class="anchor">Part f) Critical evaluation of the various algorithms </h3>
<p>
After all these glorious calculations, you should now summarize the various algorithms and come with a critical evaluation of their pros and cons. Which algorithm works best for the regression case and which is best for the classification case. These codes will also be part of your final project 3, but now applied to other data sets.
<h2 id="___sec7" class="anchor">Background literature </h2>
<ol>
<li> The text of Michael Nielsen is highly recommended, see <a href="http://neuralnetworksanddeeplearning.com/" target="_self">Nielsen's book</a>. It is an excellent read.</li>
<li> The textbook of <a href="https://www.springer.com/gp/book/9780387848570" target="_self">Trevor Hastie, Robert Tibshirani, Jerome H. Friedman, The Elements of Statistical Learning, Springer</a>, chapters 3 and 7 are the most relevant ones for the analysis here.</li>
<li> <a href="https://arxiv.org/abs/1803.08823" target="_self">Mehta et al, arXiv 1803.08823</a>, <em>A high-bias, low-variance introduction to Machine Learning for physicists</em>, ArXiv:1803.08823.</li>
</ol>
If you wish to read more about the Ising model and statistical physics here are three suggestions.
<ol>
<li> <a href="http://www.worldscientific.com/worldscibooks/10.1142/5660" target="_self">M. Plischke and B. Bergersen</a>, <em>Equilibrium Statistical Physics</em>, World Scientific, see chapters 5 and 6.</li>
<li> <a href="http://www.cambridge.org/no/academic/subjects/physics/computational-science-and-modelling/guide-monte-carlo-simulations-statistical-physics-4th-edition?format=HB" target="_self">D. P. Landau and K. Binder</a>, <em>A Guide to Monte Carlo Simulations in Statistical Physics</em>, Cambridge, see chapters 2,3 and 4.</li>
<li> <a href="https://global.oup.com/academic/product/monte-carlo-methods-in-statistical-physics-9780198517979?cc=no&lang=en&" target="_self">M. E. J. Newman and T. Barkema</a>, <em>Monte Carlo Methods in Statistical Physics</em>, Oxford, see chapters 3 and 4.</li>
</ol>
<h2 id="___sec8" class="anchor">Introduction to numerical projects </h2>
<p>
Here follows a brief recipe and recommendation on how to write a report for each
project.
<ul>
<li> Give a short description of the nature of the problem and the eventual numerical methods you have used.</li>
<li> Describe the algorithm you have used and/or developed. Here you may find it convenient to use pseudocoding. In many cases you can describe the algorithm in the program itself.</li>
<li> Include the source code of your program. Comment your program properly.</li>
<li> If possible, try to find analytic solutions, or known limits in order to test your program when developing the code.</li>
<li> Include your results either in figure form or in a table. Remember to label your results. All tables and figures should have relevant captions and labels on the axes.</li>
<li> Try to evaluate the reliabilty and numerical stability/precision of your results. If possible, include a qualitative and/or quantitative discussion of the numerical stability, eventual loss of precision etc.</li>
<li> Try to give an interpretation of you results in your answers to the problems.</li>
<li> Critique: if possible include your comments and reflections about the exercise, whether you felt you learnt something, ideas for improvements and other thoughts you've made when solving the exercise. We wish to keep this course at the interactive level and your comments can help us improve it.</li>
<li> Try to establish a practice where you log your work at the computerlab. You may find such a logbook very handy at later stages in your work, especially when you don't properly remember what a previous test version of your program did. Here you could also record the time spent on solving the exercise, various algorithms you may have tested or other topics which you feel worthy of mentioning.</li>
</ul>
<h2 id="___sec9" class="anchor">Format for electronic delivery of report and programs </h2>
<p>
The preferred format for the report is a PDF file. You can also use DOC or postscript formats or as an ipython notebook file. As programming language we prefer that you choose between C/C++, Fortran2008 or Python. The following prescription should be followed when preparing the report:
<ul>
<li> Use Devilry to hand in your projects, log in at <a href="http://devilry.ifi.uio.no" target="_self"><tt>http://devilry.ifi.uio.no</tt></a> with your normal UiO username and password and choose either 'fysstk3155' or 'fysstk4155'. There you can load up the files within the deadline.</li>
<li> Upload <b>only</b> the report file! For the source code file(s) you have developed please provide us with your link to your github domain. The report file should include all of your discussions and a list of the codes you have developed. Do not include library files which are available at the course homepage, unless you have made specific changes to them.</li>
<li> In your git repository, please include a folder which contains selected results. These can be in the form of output from your code for a selected set of runs and input parameters.</li>
<li> In this and all later projects, you should include tests (for example unit tests) of your code(s).</li>
<li> Comments from us on your projects, approval or not, corrections to be made etc can be found under your Devilry domain and are only visible to you and the teachers of the course.</li>
</ul>
Finally,
we encourage you to collaborate. Optimal working groups consist of
2-3 students. You can then hand in a common report.
<h2 id="___sec10" class="anchor">Software and needed installations </h2>
<p>
If you have Python installed (we recommend Python3) and you feel pretty familiar with installing different packages,
we recommend that you install the following Python packages via <b>pip</b> as
<ol>
<li> pip install numpy scipy matplotlib ipython scikit-learn tensorflow sympy pandas pillow</li>
</ol>
For Python3, replace <b>pip</b> with <b>pip3</b>.
<p>
See below for a discussion of <b>tensorflow</b> and <b>scikit-learn</b>.
<p>
For OSX users we recommend also, after having installed Xcode, to install <b>brew</b>. Brew allows
for a seamless installation of additional software via for example
<ol>
<li> brew install python3</li>
</ol>
For Linux users, with its variety of distributions like for example the widely popular Ubuntu distribution
you can use <b>pip</b> as well and simply install Python as
<ol>
<li> sudo apt-get install python3 (or python for python2.7)</li>
</ol>
etc etc.
<p>
If you don't want to install various Python packages with their dependencies separately, we recommend two widely used distrubutions which set up all relevant dependencies for Python, namely
<ol>
<li> <a href="https://docs.anaconda.com/" target="_self">Anaconda</a> Anaconda is an open source distribution of the Python and R programming languages for large-scale data processing, predictive analytics, and scientific computing, that aims to simplify package management and deployment. Package versions are managed by the package management system <b>conda</b></li>
<li> <a href="https://www.enthought.com/product/canopy/" target="_self">Enthought canopy</a> is a Python distribution for scientific and analytic computing distribution and analysis environment, available for free and under a commercial license.</li>
</ol>
Popular software packages written in Python for ML are
<ul>
<li> <a href="http://scikit-learn.org/stable/" target="_self">Scikit-learn</a>,</li>
<li> <a href="https://www.tensorflow.org/" target="_self">Tensorflow</a>,</li>
<li> <a href="http://pytorch.org/" target="_self">PyTorch</a> and</li>
<li> <a href="https://keras.io/" target="_self">Keras</a>.</li>
</ul>
These are all freely available at their respective GitHub sites. They
encompass communities of developers in the thousands or more. And the number
of code developers and contributors keeps increasing.
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li class="active"><a href="._Project2-bs000.html">1</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright --> &copy; 1999-2018, "Data Analysis and Machine Learning FYS-STK3155/FYS4155":"http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html". Released under CC Attribution-NonCommercial 4.0 license
</center>
</body>
</html>
doc/Projects/2018/Project2/pdf/Project2.tex~
+557
View File
@@ -0,0 +1,557 @@
%%
%% Automatically generated file from DocOnce source
%% (https://github.com/hplgit/doconce/)
%%
%%
%-------------------- begin preamble ----------------------
\documentclass[%
oneside, % oneside: electronic viewing, twoside: printing
final, % draft: marks overfull hboxes, figures with paths
10pt]{article}
\listfiles % print all files needed to compile this document
\usepackage{relsize,makeidx,color,setspace,amsmath,amsfonts,amssymb}
\usepackage[table]{xcolor}
\usepackage{bm,ltablex,microtype}
\usepackage[pdftex]{graphicx}
\usepackage{fancyvrb} % packages needed for verbatim environments
\usepackage[T1]{fontenc}
%\usepackage[latin1]{inputenc}
\usepackage{ucs}
\usepackage[utf8x]{inputenc}
\usepackage{lmodern} % Latin Modern fonts derived from Computer Modern
% Hyperlinks in PDF:
\definecolor{linkcolor}{rgb}{0,0,0.4}
\usepackage{hyperref}
\hypersetup{
breaklinks=true,
colorlinks=true,
linkcolor=linkcolor,
urlcolor=linkcolor,
citecolor=black,
filecolor=black,
%filecolor=blue,
pdfmenubar=true,
pdftoolbar=true,
bookmarksdepth=3 % Uncomment (and tweak) for PDF bookmarks with more levels than the TOC
}
%\hyperbaseurl{} % hyperlinks are relative to this root
\setcounter{tocdepth}{2} % levels in table of contents
% --- fancyhdr package for fancy headers ---
\usepackage{fancyhdr}
\fancyhf{} % sets both header and footer to nothing
\renewcommand{\headrulewidth}{0pt}
\fancyfoot[LE,RO]{\thepage}
% Ensure copyright on titlepage (article style) and chapter pages (book style)
\fancypagestyle{plain}{
\fancyhf{}
\fancyfoot[C]{{\footnotesize \copyright\ 1999-2018, "Data Analysis and Machine Learning FYS-STK3155/FYS4155":"http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html". Released under CC Attribution-NonCommercial 4.0 license}}
% \renewcommand{\footrulewidth}{0mm}
\renewcommand{\headrulewidth}{0mm}
}
% Ensure copyright on titlepages with \thispagestyle{empty}
\fancypagestyle{empty}{
\fancyhf{}
\fancyfoot[C]{{\footnotesize \copyright\ 1999-2018, "Data Analysis and Machine Learning FYS-STK3155/FYS4155":"http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html". Released under CC Attribution-NonCommercial 4.0 license}}
\renewcommand{\footrulewidth}{0mm}
\renewcommand{\headrulewidth}{0mm}
}
\pagestyle{fancy}
% prevent orhpans and widows
\clubpenalty = 10000
\widowpenalty = 10000
% --- end of standard preamble for documents ---
% insert custom LaTeX commands...
\raggedbottom
\makeindex
\usepackage[totoc]{idxlayout} % for index in the toc
\usepackage[nottoc]{tocbibind} % for references/bibliography in the toc
%-------------------- end preamble ----------------------
\begin{document}
% matching end for #ifdef PREAMBLE
\newcommand{\exercisesection}[1]{\subsection*{#1}}
% ------------------- main content ----------------------
% ----------------- title -------------------------
\thispagestyle{empty}
\begin{center}
{\LARGE\bf
\begin{spacing}{1.25}
Project 2 on Machine Learning, deadline November 12
\end{spacing}
}
\end{center}
% ----------------- author(s) -------------------------
\begin{center}
{\bf \href{{http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html}}{Data Analysis and Machine Learning FYS-STK3155/FYS4155}}
\end{center}
\begin{center}
% List of all institutions:
\centerline{{\small Department of Physics, University of Oslo, Norway}}
\end{center}
% ----------------- end author(s) -------------------------
% --- begin date ---
\begin{center}
Nov 2, 2018
\end{center}
% --- end date ---
\vspace{1cm}
\subsection*{Classification and Regression, from linear and logistic regression to neural networks}
The main aim of this project is to study both classification and
regression problems, starting with the regression algorithms studied
in project 1. We will include logistic regresion for classification
problems and write our own multilayer perceptron code for studying
both regression and classification problems. The codes developed in
project 1, including bootstrap and/or cross-validation as well as the
computation of the mean-squared error and the R2 score function can
also be utilized (and included in logistic regression and the neural
network codes) in the present analysis.
We will use the so-called Ising model for our training data and will
focus on supervised training. We will follow closely the recent
article of \href{{https://arxiv.org/abs/1803.08823}}{Mehta et al, arXiv
1803.08823}. This article stands
out as an excellent review on machine learning (ML) algorithms.
The added benefit is that each figure and
model presented in \href{{https://physics.bu.edu/~pankajm/MLnotebooks.html}}{this article is accompanied by its jupyter
notebook}. This
means that we can start using these and compare with our own results.
They provide also the data set for the regression and classification
analysis that we will explore. In this sense, with their available
notebooks, it makes life easier since we can compare our own codes
with their codes.
With the abovementioned configurations we will determine, using first
various regression methods, the value of the coupling constant for the
energy of the one-dimensional Ising model. Thereafter, we will use the
two-dimensional data, but now computed at different temperatures, in
order to classify the phase of the Ising model. Below the critical
temperature, the system will be in a so-called ferromagnetic
phase. Close to the critical temperature, the final magnetization
becomes smaller and smaller in absolute value while above the critical
temperature, the net magnetization is zero. This classification case,
that is the two-dimensional Ising model, will be studied using
logistic regression and deep neural networks. The aim is to develop
your own logistic regression code for the classification of the phases
(this is a binary model) and your multilayer perceptron code for the
classification and regression case. You can compare your own results with those obtained
using \textbf{scikit-learn} or \textbf{tensorflow} or other Python packages such as \textbf{keras} or other.
Feel free to use the notebooks to benchmark your code. If you wish to
write your own C++ or Fortran program for say a multilayer neural network
model and a logistic regression model, please feel free to do so.
\paragraph{Part a): Producing the data for the one-dimensional Ising model.}
The model we will employ in our studies is the so-called \href{{https://en.wikipedia.org/wiki/Ising_model}}{Ising
model}. Together with
models like the \href{{https://en.wikipedia.org/wiki/Potts_model}}{Potts
model} and similar
so-called lattice models, the Ising model has been widely studied in
mathematics (in statistics in particular), physics, \href{{https://journals.aps.org/pre/abstract/10.1103/PhysRevE.93.062402}}{life
science},
chemistry and even in the \href{{https://www.springer.com/gp/book/9781461420316}}{social sciences in order to model social
behavior}. It is a
simple binary value system where the variables of the model (spins often in
physics) can take two values only, for example $\pm 1$ or $0$ and $1$.
The system exhibits a phase transition in two or higher dimensions and
the first person to find the analytical expressions for various
expectation values was the Norwegian chemist \href{{https://en.wikipedia.org/wiki/Lars_Onsager}}{Lars
Onsager} (Nobel prize in
chemistry) after a tour de force mathematics exercise.
In our discussions here we will stay with a physicist's approach and
call the variables for spin. You could replace this with any other
type of binary variables, ranging from a two political parties to blue
and red spheres. In its simplest form we define the energy of the
system as
\begin{equation*}
E=-J\sum_{<kl>}^{N}s_ks_l,
\end{equation*}
with $s_k=\pm 1$, $N$ is the total number of spins,
$J$ is a coupling constant expressing the strength of the interaction
between neighboring spins.
The symbol $<kl>$ indicates that we sum over nearest
neighbors only.
Notice that for $J>0$ it is energetically favorable for neighboring spins
to be aligned. This feature leads to, at low enough temperatures,
a cooperative phenomenon called spontaneous magnetization. That is,
through interactions between nearest neighbors, a given magnetic
moment can influence the alignment of spins that are separated
from the given spin by a macroscopic distance. These long range correlations
between spins are associated with a long-range order in which
the lattice has a net magnetization in the absence of a magnetic field.
We start by considering the one-dimensional Ising model with nearest neighbor interactions. This model does not exhibit any phase transition.
Consider the 1D Ising model with nearest-neighbor interactions
\begin{equation*}
E[\hat{s}]=-J\sum_{j=1}^{N}s_{j}s_{j+1},
\end{equation*}
on a chain of length $N$ with so-called periodic boundary conditions and $S_j=\pm 1$ Ising spin variables.
In one dimension, this model has no phase transition at finite temperature.
In the Python code below we generate, with a coupling coefficient set to $J=1$, a large number of spin configurations say $10000$ as shown in the code below.
It means that our data will be a set of $i=1\ldots n$ points of the form
$\{(E[\boldsymbol{s}^i],\boldsymbol{s}^i)\}$.
Our task is to find the value of $J$ from the data set using linear regression.
Here is the Python code you need to generate the training data, see
also the \href{{https://physics.bu.edu/~pankajm/ML-Notebooks/HTML/NB_CVI-linreg_ising.html}}{notebook of Mehta et
al}.
\begin{print}
import numpy as np
import scipy.sparse as sp
np.random.seed(12)
import warnings
#Comment this to turn on warnings
warnings.filterwarnings('ignore')
### define Ising model aprams
# system size
L=40
# create 10000 random Ising states
states=np.random.choice([-1, 1], size=(10000,L))
def ising_energies(states,L):
"""
This function calculates the energies of the states in the nn Ising Hamiltonian
"""
J=np.zeros((L,L),)
for i in range(L):
J[i,(i+1)%L]-=1.0
# compute energies
E = np.einsum('...i,ij,...j->...',states,J,states)
return E
# calculate Ising energies
energies=ising_energies(states,L)
\end{print}
We can now recast the problem as a linear regression model using our codes from project 1.
The way we are going to build our model mimicks the way we could think of finding say the gravitional constant for the graviational force between two planets.
In the absence of any prior knowledge, one sensible choice is the all-to-all Ising model
\[
E_\mathrm{model}[\boldsymbol{s}^i] = - \sum_{j=1}^N \sum_{k=1}^N J_{j,k}s_{j}^is_{k}^i.
\]
Here $i$ represents a particular spin configuration (one of the possible $n$ configurations we generated with the code above).
This model is uniquely defined by the non-local coupling strengths $J_{jk}$ which we want to learn.
The model is linear in $\mathbf{J}$ which makes it possible to use linear regression.
To apply linear regression, we recast this model in the form
\[
E_\mathrm{model}^i \equiv \mathbf{X}^i \cdot \mathbf{J},
\]
where the vectors $\mathbf{X}^i$ represent all two-body interactions
$\{s_{j}^is_{k}^i \}_{j,k=1}^N$, and the index $i$ runs over the
samples in the data set. To make the analogy complete, we can also
represent the dot product by a single index $p = \{j,k\}$,
i.e.~$\mathbf{X}^i \cdot \mathbf{J}=X^i_pJ_p$. Note that the
regression model does not include the minus sign, so we expect to
learn negative $J$'s.
With these preliminaries, we are now ready to reutilize our codes from project 1.
\paragraph{Part b): Estimating the coupling constant of the one-dimensional Ising model using linear regression.}
We start with the one-dimensional Ising model and use the data we have
generated with $J=1$ in the previous point. Use linear regression,
Lasso and Ridge regression as done in project 1. You can compare your
results with those of \href{{https://physics.bu.edu/~pankajm/ML-Notebooks/HTML/NB_CVI-linreg_ising.html}}{Mehta
et al.}.
Make sure it is the 1D data which is used.
Discuss the methods and how they perform in computing the coupling
constant $J$ and include a bias-variance analysis using either
cross-validation or bootstrap. Discuss also the mean squared error and
the $R2$ score as measures to assess your model.
Give a critical analysis of your results.
\paragraph{Part c): Determine the phase of the two-dimensional Ising model.}
We switch now to binary classification methods and use logistic
regression to define the phases of the Ising model. This means that we switch to the two-dimensional Ising model
and use the data sets generated by \href{{https://physics.bu.edu/~pankajm/ML-Review-Datasets/isingMC/}}{Mehta et al}
These energies and their corresponding spin orientation configurations
represent then your data. We will use a fixed lattice of $L\times L =
40 \times 40$ spins in two dimensions. The link above contains data for several temperatures.
The theoretical critical temperature for a phase transition is $T_C\approx 2.269$ in units of energy.
However, for a finite lattice the results representing the critical temperature are slightly higher ($T_C \approx 2.3$).
Our goal here, using logistic regression, is to train our model to
predict the phase of a sample given the spin configuration, whether it
represents a state above the critical temperature or below. The
configurations representing states below the critical temperature are
called ordered states (the spins tend to point in one direction,
resulting in a net magnetic moment) while those above the critical
temperature are called disordered. Since a finite lattice like this
does not exhibit a clear sign of a phase transition we will mainly
stay with either orderer or disordered phases. You could include the
critical phase if you want.
Your aim here is thus to read in these data (use the examples from
\href{{https://physics.bu.edu/~pankajm/ML-Notebooks/HTML/NB_CVII-logreg_ising.html}}{Mehta et
al})
and write your own code for doing logistic regression, see the lecture
notes on \href{{https://compphysics.github.io/MachineLearning/doc/pub/LogReg/html/LogReg-bs.html}}{logistic
regression}.
In this case, to evaluate the model, we will use the so-called accuracy score
instead of the bootstrap or cross-validation as done in the standard linear regression part discussed in b). Examples of how to define the accuracy score can be found under the neural network slides, see for example the \href{{https://compphysics.github.io/MachineLearning/doc/pub/NeuralNet/html/._NeuralNet-bs047.html}}{slides here}.
To measure the performance of our network we evaluate how well it does
it data it has never seen before, i.e.~the test data. We measure the
performance of the network using the \emph{accuracy} score. The accuracy
is as you would expect just the number of images correctly labeled
divided by the total number of images. A perfect classifier will have
an accuracy score of $1$.
\[
\text{Accuracy} = \frac{\sum_{i=1}^n I(t_i = y_i)}{n} ,
\]
where $I$ is the indicator function, $1$ if $t_i = y_i$ and $0$
otherwise, where $t_i$ represents the target and $y_i$ the outputs.
In order to find the optimal parameters of your logistic regressor you should
include a gradient descent solver, as discussed in the \href{{https://compphysics.github.io/MachineLearning/doc/pub/Splines/html/Splines-bs.html}}{gradient
descent
lectures}.
Since we don't have so many data points, you may just code the
standard gradient descent with a given learning rate, or even attempt
to use the Newton-Raphson method. Alternatively, it may be useful for
the next part on neural networks to implement a stochastic gradient
descent with and without mini-batches. Stochastic gradient with mini-batches may give the best results. You could finally compare your code with the output from \textbf{scikit-learn}'s toolbox for
optimization methods applied to logistic regression.
The notebook of \href{{https://physics.bu.edu/~pankajm/ML-Notebooks/HTML/NB_CVII-logreg_ising.html}}{Mehta et al} is highly recommended in order to benchmark your code and results.
\paragraph{Part d): Regression analysis of the one-dimensional Ising model using neural networks.}
Your aim now, and this is the central part of this project, is to
write to your own multilayer perceptron model implementing the back
propagation algorithm discussed in the \href{{https://compphysics.github.io/MachineLearning/doc/pub/NeuralNet/html/NeuralNet-bs.html}}{lecture
slides}. We
start with the regression case discussed in parts a) and b) but train
now the network to find the optimal weights and biases. You are free
to use the codes in the above lecture slides as starting points.
Train your network and compare the results with those from your linear regression code.
You can test your results against a similar code using _scikit_learn_ (see the examples in the above lecture notes) or \textbf{tensorflow/keras}.
A useful reference on the back progagation algorithm is \href{{http://neuralnetworksanddeeplearning.com/}}{Nielsen's book}. It is an excellent read.
\paragraph{Part e): Classifying the Ising model phase using neural networks.}
Finally, change now your cost function to the $log$ cross-entropy classification cost function for the case discussed in part c). Train your network again and
compare the results with those from your logistic regression code i c).
Here again you can compare your results with those of \href{{https://physics.bu.edu/~pankajm/ML-Notebooks/HTML/NB_CIX-DNN_ising_TFlow.html}}{Mehta et al}. There they used \textbf{tensorflow} to classify the phases.
\paragraph{Part f) Critical evaluation of the various algorithms.}
After all these glorious calculations, you should now summarize the various algorithms and come with a critical evaluation of their pros and cons. Which algorithm works best for the regression case and which is best for the classification case. These codes will also be part of your final project 3, but now applied to other data sets.
\subsection*{Background literature}
\begin{enumerate}
\item The text of Michael Nielsen is highly recommended, see \href{{http://neuralnetworksanddeeplearning.com/}}{Nielsen's book}. It is an excellent read.
\item The textbook of \href{{https://www.springer.com/gp/book/9780387848570}}{Trevor Hastie, Robert Tibshirani, Jerome H. Friedman, The Elements of Statistical Learning, Springer}, chapters 3 and 7 are the most relevant ones for the analysis here.
\item \href{{https://arxiv.org/abs/1803.08823}}{Mehta et al, arXiv 1803.08823}, \emph{A high-bias, low-variance introduction to Machine Learning for physicists}, ArXiv:1803.08823.
\end{enumerate}
\noindent
If you wish to read more about the Ising model and statistical physics here are three suggestions.
\begin{enumerate}
\item \href{{http://www.worldscientific.com/worldscibooks/10.1142/5660}}{M. Plischke and B. Bergersen}, \emph{Equilibrium Statistical Physics}, World Scientific, see chapters 5 and 6.
\item \href{{http://www.cambridge.org/no/academic/subjects/physics/computational-science-and-modelling/guide-monte-carlo-simulations-statistical-physics-4th-edition?format=HB}}{D. P. Landau and K. Binder}, \emph{A Guide to Monte Carlo Simulations in Statistical Physics}, Cambridge, see chapters 2,3 and 4.
\item \href{{https://global.oup.com/academic/product/monte-carlo-methods-in-statistical-physics-9780198517979?cc=no&lang=en&}}{M. E. J. Newman and T. Barkema}, \emph{Monte Carlo Methods in Statistical Physics}, Oxford, see chapters 3 and 4.
\end{enumerate}
\noindent
\subsection*{Introduction to numerical projects}
Here follows a brief recipe and recommendation on how to write a report for each
project.
\begin{itemize}
\item Give a short description of the nature of the problem and the eventual numerical methods you have used.
\item Describe the algorithm you have used and/or developed. Here you may find it convenient to use pseudocoding. In many cases you can describe the algorithm in the program itself.
\item Include the source code of your program. Comment your program properly.
\item If possible, try to find analytic solutions, or known limits in order to test your program when developing the code.
\item Include your results either in figure form or in a table. Remember to label your results. All tables and figures should have relevant captions and labels on the axes.
\item Try to evaluate the reliabilty and numerical stability/precision of your results. If possible, include a qualitative and/or quantitative discussion of the numerical stability, eventual loss of precision etc.
\item Try to give an interpretation of you results in your answers to the problems.
\item Critique: if possible include your comments and reflections about the exercise, whether you felt you learnt something, ideas for improvements and other thoughts you've made when solving the exercise. We wish to keep this course at the interactive level and your comments can help us improve it.
\item Try to establish a practice where you log your work at the computerlab. You may find such a logbook very handy at later stages in your work, especially when you don't properly remember what a previous test version of your program did. Here you could also record the time spent on solving the exercise, various algorithms you may have tested or other topics which you feel worthy of mentioning.
\end{itemize}
\noindent
\subsection*{Format for electronic delivery of report and programs}
The preferred format for the report is a PDF file. You can also use DOC or postscript formats or as an ipython notebook file. As programming language we prefer that you choose between C/C++, Fortran2008 or Python. The following prescription should be followed when preparing the report:
\begin{itemize}
\item Use Devilry to hand in your projects, log in at \href{{http://devilry.ifi.uio.no}}{\nolinkurl{http://devilry.ifi.uio.no}} with your normal UiO username and password and choose either 'fysstk3155' or 'fysstk4155'. There you can load up the files within the deadline.
\item Upload \textbf{only} the report file! For the source code file(s) you have developed please provide us with your link to your github domain. The report file should include all of your discussions and a list of the codes you have developed. Do not include library files which are available at the course homepage, unless you have made specific changes to them.
\item In your git repository, please include a folder which contains selected results. These can be in the form of output from your code for a selected set of runs and input parameters.
\item In this and all later projects, you should include tests (for example unit tests) of your code(s).
\item Comments from us on your projects, approval or not, corrections to be made etc can be found under your Devilry domain and are only visible to you and the teachers of the course.
\end{itemize}
\noindent
Finally,
we encourage you to collaborate. Optimal working groups consist of
2-3 students. You can then hand in a common report.
\subsection*{Software and needed installations}
If you have Python installed (we recommend Python3) and you feel pretty familiar with installing different packages,
we recommend that you install the following Python packages via \textbf{pip} as
\begin{enumerate}
\item pip install numpy scipy matplotlib ipython scikit-learn tensorflow sympy pandas pillow
\end{enumerate}
\noindent
For Python3, replace \textbf{pip} with \textbf{pip3}.
See below for a discussion of \textbf{tensorflow} and \textbf{scikit-learn}.
For OSX users we recommend also, after having installed Xcode, to install \textbf{brew}. Brew allows
for a seamless installation of additional software via for example
\begin{enumerate}
\item brew install python3
\end{enumerate}
\noindent
For Linux users, with its variety of distributions like for example the widely popular Ubuntu distribution
you can use \textbf{pip} as well and simply install Python as
\begin{enumerate}
\item sudo apt-get install python3 (or python for python2.7)
\end{enumerate}
\noindent
etc etc.
If you don't want to install various Python packages with their dependencies separately, we recommend two widely used distrubutions which set up all relevant dependencies for Python, namely
\begin{enumerate}
\item \href{{https://docs.anaconda.com/}}{Anaconda} Anaconda is an open source distribution of the Python and R programming languages for large-scale data processing, predictive analytics, and scientific computing, that aims to simplify package management and deployment. Package versions are managed by the package management system \textbf{conda}
\item \href{{https://www.enthought.com/product/canopy/}}{Enthought canopy} is a Python distribution for scientific and analytic computing distribution and analysis environment, available for free and under a commercial license.
\end{enumerate}
\noindent
Popular software packages written in Python for ML are
\begin{itemize}
\item \href{{http://scikit-learn.org/stable/}}{Scikit-learn},
\item \href{{https://www.tensorflow.org/}}{Tensorflow},
\item \href{{http://pytorch.org/}}{PyTorch} and
\item \href{{https://keras.io/}}{Keras}.
\end{itemize}
\noindent
These are all freely available at their respective GitHub sites. They
encompass communities of developers in the thousands or more. And the number
of code developers and contributors keeps increasing.
% ------------------- end of main content ---------------
\end{document}
doc/Projects/2018/hw1/html/._hw1-bs000.html
+285
View File
@@ -0,0 +1,285 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="description" content="Homework 1">
<title>Homework 1</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Exercise 1', 2, None, '___sec0'),
('Exercise 2', 2, None, '___sec1'),
('Exercise 3, variance of the parameters $\\beta$ in linear '
'regression',
2,
None,
'___sec2')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="hw1-bs.html">Homework 1</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="#___sec0" style="font-size: 80%;">Exercise 1</a></li>
<!-- navigation toc: --> <li><a href="#___sec1" style="font-size: 80%;">Exercise 2</a></li>
<!-- navigation toc: --> <li><a href="#___sec2" style="font-size: 80%;">Exercise 3, variance of the parameters \( \beta \) in linear regression</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0000"></a>
<!-- ------------------- main content ---------------------- -->
<div class="jumbotron">
<center><h1>Homework 1</h1></center> <!-- document title -->
<p>
<!-- author(s): <a href="http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html" target="_self">Data Analysis and Machine Learning FYS-STK3155/FYS4155</a> -->
<center>
<b><a href="http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html" target="_self">Data Analysis and Machine Learning FYS-STK3155/FYS4155</a></b>
</center>
<p>
<!-- institution -->
<center><b>Department of Physics, University of Oslo, Norway</b></center>
<br>
<p>
<center><h4>Aug 30, 2018</h4></center> <!-- date -->
<br>
<p>
</div> <!-- end jumbotron -->
<h2 id="___sec0" class="anchor">Exercise 1 </h2>
<p>
The first exercise here is of a mere technical art. We want you to have
<ul>
<li> git as a version control software and to establish a user account on a provider like GitHub. Other providers like GitLab etc are equally fine. You can also use the University of Oslo <a href="https://www.uio.no/tjenester/it/maskin/filer/versjonskontroll/github.html" target="_self">GitHub facilities</a>.</li>
<li> Install various Python packages</li>
</ul>
We will make extensive use of Python as programming language and its
myriad of available libraries. You will find
IPython/Jupyter notebooks invaluable in your work. You can run <b>R</b>
codes in the Jupyter/IPython notebooks, with the immediate benefit of
visualizing your data. You can also use compiled languages like C++,
Rust, Fortran etc if you prefer. The focus in these lectures will be
on Python.
<p>
If you have Python installed (we recommend Python3) and you feel
pretty familiar with installing different packages, we recommend that
you install the following Python packages via <b>pip</b> as
<ol>
<li> pip install numpy scipy matplotlib ipython scikit-learn sympy pandas pillow</li>
</ol>
For <b>Tensorflow</b>, we recommend following the instructions in the text of
<a href="http://shop.oreilly.com/product/0636920052289.do" target="_self">Aurelien Geron, Hands&#8209;On Machine Learning with Scikit&#8209;Learn and TensorFlow, O'Reilly</a>
<p>
We will come back to <b>tensorflow</b> later.
<p>
For Python3, replace <b>pip</b> with <b>pip3</b>.
<p>
For OSX users we recommend, after having installed Xcode, to
install <b>brew</b>. Brew allows for a seamless installation of additional
software via for example
<ol>
<li> brew install python3</li>
</ol>
For Linux users, with its variety of distributions like for example the widely popular Ubuntu distribution,
you can use <b>pip</b> as well and simply install Python as
<ol>
<li> sudo apt-get install python3 (or python for pyhton2.7)</li>
</ol>
If you don't want to perform these operations separately and venture
into the hassle of exploring how to set up dependencies and paths, we
recommend two widely used distrubutions which set up all relevant
dependencies for Python, namely
<ul>
<li> <a href="https://docs.anaconda.com/" target="_self">Anaconda</a>,</li>
</ul>
which is an open source
distribution of the Python and R programming languages for large-scale
data processing, predictive analytics, and scientific computing, that
aims to simplify package management and deployment. Package versions
are managed by the package management system <b>conda</b>.
<ul>
<li> <a href="https://www.enthought.com/product/canopy/" target="_self">Enthought canopy</a></li>
</ul>
is a Python
distribution for scientific and analytic computing distribution and
analysis environment, available for free and under a commercial
license.
<p>
We recommend using <b>Anaconda</b>.
<h2 id="___sec1" class="anchor">Exercise 2 </h2>
<p>
We will generate our own dataset for a function \( y(x) \) where \( x \in [0,1] \) and defined by random numbers computed with the uniform distribution. The function \( y \) is a quadratic polynomial in \( x \) with added stochastic noise according to the normal distribution \( \cal {N}(0,1) \).
The following simple Python instructions define our \( x \) and \( y \) values (with 100 data points).
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>x <span style="color: #666666">=</span> np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>rand(<span style="color: #666666">100</span>,<span style="color: #666666">1</span>)
y <span style="color: #666666">=</span> <span style="color: #666666">5*</span>x<span style="color: #666666">*</span>x<span style="color: #666666">+0.1*</span>np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>randn(<span style="color: #666666">100</span>,<span style="color: #666666">1</span>)
</pre></div>
<ol>
<li> Write your own code (following the examples under the <a href="https://compphysics.github.io/MachineLearning/doc/pub/Regression/html/Regression-bs.html" target="_self">regression slides</a>) for computing the parametrization of the data set fitting a second-order polynomial.</li>
<li> Use thereafter <b>scikit-learn</b> (see again the examples in the regression slides) and compare with your own code.</li>
<li> Using scikit-learn, compute also the mean square error, a risk metric corresponding to the expected value of the squared (quadratic) error defined as</li>
</ol>
$$ MSE(\hat{y},\hat{\tilde{y}}) = \frac{1}{n}
\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2,
$$
and the \( R^2 \) score function.
If \( \tilde{\hat{y}}_i \) is the predicted value of the \( i-th \) sample and \( y_i \) is the corresponding true value, then the score \( R^2 \) is defined as
$$
R^2(\hat{y}, \tilde{\hat{y}}) = 1 - \frac{\sum_{i=0}^{n - 1} (y_i - \tilde{y}_i)^2}{\sum_{i=0}^{n - 1} (y_i - \bar{y})^2},
$$
where we have defined the mean value of \( \hat{y} \) as
$$
\bar{y} = \frac{1}{n} \sum_{i=0}^{n - 1} y_i.
$$
You can use the functionality included in scikit-learn. If you feel for it, you can use your own program and define functions which compute the above two functions.
Discuss the meaning of these results. Try also to vary the coefficient in front of the added stochastic noise term and discuss the quality of the fits.
<h2 id="___sec2" class="anchor">Exercise 3, variance of the parameters \( \beta \) in linear regression </h2>
<p>
Show that the variance of the parameters \( \beta \) in the linear regression method (chapter 3, equation (3.8) of <a href="https://www.springer.com/gp/book/9780387848570" target="_self">Trevor Hastie, Robert Tibshirani, Jerome H. Friedman, The Elements of Statistical Learning, Springer</a>) is given as
$$
\mathrm{Var}(\hat{\beta}) = \left(\hat{X}^T\hat{X}\right)^{-1}\sigma^2,
$$
with
$$
\sigma^2 = \frac{1}{N-p-1}\sum_{i=1}^{N} (y_i-\tilde{y}_i)^2,
$$
where we have assumed that we fit a function of degree \( p-1 \) (for example a polynomial in \( x \)).
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li class="active"><a href="._hw1-bs000.html">1</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright --> &copy; 1999-2018, "Data Analysis and Machine Learning FYS-STK3155/FYS4155":"http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html". Released under CC Attribution-NonCommercial 4.0 license
</center>
</body>
</html>
doc/Projects/2018/hw2/html/._hw2-bs000.html
+211
View File
@@ -0,0 +1,211 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="description" content="Homework 2">
<title>Homework 2</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Exercise 4', 2, None, '___sec0'),
('Exercise 5', 2, None, '___sec1')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="hw2-bs.html">Homework 2</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="#___sec0" style="font-size: 80%;">Exercise 4</a></li>
<!-- navigation toc: --> <li><a href="#___sec1" style="font-size: 80%;">Exercise 5</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0000"></a>
<!-- ------------------- main content ---------------------- -->
<div class="jumbotron">
<center><h1>Homework 2</h1></center> <!-- document title -->
<p>
<!-- author(s): <a href="http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html" target="_self">Data Analysis and Machine Learning FYS-STK3155/FYS4155</a> -->
<center>
<b><a href="http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html" target="_self">Data Analysis and Machine Learning FYS-STK3155/FYS4155</a></b>
</center>
<p>
<!-- institution -->
<center><b>Department of Physics, University of Oslo, Norway</b></center>
<br>
<p>
<center><h4>Sep 5, 2018</h4></center> <!-- date -->
<br>
<p>
</div> <!-- end jumbotron -->
<h2 id="___sec0" class="anchor">Exercise 4 </h2>
<p>
This exercise is a continuation of exercise 2 from homework 1. We will
use the same function to generate our data set, still staying with a
simple function \( y(x) \) which we want to fit using linear regression,
but now extending the analysis to include the Ridge and the Lasso
regression methods. You can use the code under the Regression as an example on how to use the Ridge and the Lasso methods, see the <a href="https://compphysics.github.io/MachineLearning/doc/pub/Regression/html/Regression-bs.html" target="_self">regression slides</a>).
<p>
We will thus again generate our own dataset for a function \( y(x) \) where
\( x \in [0,1] \) and defined by random numbers computed with the uniform
distribution. The function \( y \) is a quadratic polynomial in \( x \) with
added stochastic noise according to the normal distribution \( \cal{N}(0,1) \).
<p>
The following simple Python instructions define our \( x \) and \( y \) values (with 100 data points).
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>x <span style="color: #666666">=</span> np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>rand(<span style="color: #666666">100</span>,<span style="color: #666666">1</span>)
y <span style="color: #666666">=</span> <span style="color: #666666">5*</span>x<span style="color: #666666">*</span>x<span style="color: #666666">+0.1*</span>np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>randn(<span style="color: #666666">100</span>,<span style="color: #666666">1</span>)
</pre></div>
<ol>
<li> Write your own code for the Ridge method (see chapter 3.4 of Hastie <em>et al.</em>, equations (3.43) and (3.44)) and compute the parametrization for different values of \( \lambda \). Compare and analyze your results with those from exercise 2. Study the dependence on \( \lambda \) while also varying the strength of the noise in your expression for \( y(x) \).</li>
<li> Repeat the above but using the functionality of <b>scikit-learn</b>. Compare your code with the results from <b>scikit-learn</b>. Remember to run with the same random numbers for generating \( x \) and \( y \).</li>
<li> Our next step is to study the variance of the parameters \( \beta_1 \) and \( \beta_2 \) (assuming that we are parametrizing our function with a second-order polynomial. We will use standard linear regression and the Ridge regression. You can now opt for either writing your own function that calculates the variance of these paramaters (recall that this is equal to the diagonal elements of the matrix \( (\hat{X}^T\hat{X})+\lambda\hat{I})^{-1} \)) or use the functionality of <b>scikit-learn</b> and compute their variances. Discuss the results of these variances as functions of \( \lambda \). In particular, try to link your discussion with the discussion in Hastie <em>et al.</em> and their figure 3.11.</li>
<li> Repeat the previous step but add now the Lasso method, see equation (3.53) of Hastie <em>et al.</em>. Discuss your results and compare with standard regression and the Ridge regression results. You can write your own code or use the functionality of <b>scikit-learn</b>.</li>
<li> Finally, using <b>scikit-learn</b> or your own code, compute also the mean square error, a risk metric corresponding to the expected value of the squared (quadratic) error defined as</li>
</ol>
$$ MSE(\hat{y},\hat{\tilde{y}}) = \frac{1}{n}
\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2,
$$
and the \( R^2 \) score function.
If \( \tilde{\hat{y}}_i \) is the predicted value of the \( i-th \) sample and \( y_i \) is the corresponding true value, then the score \( R^2 \) is defined as
$$
R^2(\hat{y}, \tilde{\hat{y}}) = 1 - \frac{\sum_{i=0}^{n - 1} (y_i - \tilde{y}_i)^2}{\sum_{i=0}^{n - 1} (y_i - \bar{y})^2},
$$
where we have defined the mean value of \( \hat{y} \) as
$$
\bar{y} = \frac{1}{n} \sum_{i=0}^{n - 1} y_i.
$$
Discuss these quantities as functions of the variable \( \lambda \) in the Ridge and Lasso regression methods.
<h2 id="___sec1" class="anchor">Exercise 5 </h2>
<p>
Using the singular value decomposition, show that the variance of the direction vector
\( \hat{z}_i=\hat{X}\hat{v}_i=\hat{u}_1d_1 \) is equal to (equation (3.49) of Hastie <em>et al.</em>)
$$
\mathrm{Var}(\hat{z}_i)=\frac{d_i^2}{N},
$$
where \( d_i \) are the singular values of the matrix \( \hat{X} \). In Hastie <em>et al</em>, the matrix elements of \( X \) are centered. The consequence is that the mean values of for example \( \hat{u}_i \) are zero.
<p>
Give an interpretation of these results, in particular in connection with the variance of the coefficients you obtained in the previous exercise.
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li class="active"><a href="._hw2-bs000.html">1</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright --> &copy; 1999-2018, "Data Analysis and Machine Learning FYS-STK3155/FYS4155":"http://www.uio.no/studier/emner/matnat/fys/FYS3155/index-eng.html". Released under CC Attribution-NonCommercial 4.0 license
</center>
</body>
</html>
doc/Textbooks/.DS_Store
Vendored
BIN
View File
Binary file not shown.
doc/pub/Autoencoders/html/._Autoencoders-bs002.html
+116
View File
@@ -0,0 +1,116 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Autoencoders">
<title>Data Analysis and Machine Learning: Autoencoders</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Autoencoders: Overarching view', 2, None, '___sec0'),
('Simple examples of Autoencoders', 2, None, '___sec1')]}
end of tocinfo -->
<body>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Autoencoders-bs.html">Data Analysis and Machine Learning: Autoencoders</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Autoencoders-bs001.html#___sec0" style="font-size: 80%;">Autoencoders: Overarching view</a></li>
<!-- navigation toc: --> <li><a href="#___sec1" style="font-size: 80%;">Simple examples of Autoencoders</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0002"></a>
<!-- !split -->
<h2 id="___sec1" class="anchor">Simple examples of Autoencoders </h2>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Autoencoders-bs001.html">&laquo;</a></li>
<li><a href="._Autoencoders-bs000.html">1</a></li>
<li><a href="._Autoencoders-bs001.html">2</a></li>
<li class="active"><a href="._Autoencoders-bs002.html">3</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/Autoencoders/html/reveal.js/plugin/leap/leap.js
+159
View File
File diff suppressed because one or more lines are too long
doc/pub/Autoencoders/html/reveal.js/plugin/remotes/remotes.js
+39
View File
@@ -0,0 +1,39 @@
/**
* Touch-based remote controller for your presentation courtesy
* of the folks at http://remotes.io
*/
(function(window){
/**
* Detects if we are dealing with a touch enabled device (with some false positives)
* Borrowed from modernizr: https://github.com/Modernizr/Modernizr/blob/master/feature-detects/touch.js
*/
var hasTouch = (function(){
return ('ontouchstart' in window) || window.DocumentTouch && document instanceof DocumentTouch;
})();
/**
* Detects if notes are enable and the current page is opened inside an /iframe
* this prevents loading Remotes.io several times
*/
var isNotesAndIframe = (function(){
return window.RevealNotes && !(self == top);
})();
if(!hasTouch && !isNotesAndIframe){
head.ready( 'remotes.ne.min.js', function() {
new Remotes("preview")
.on("swipe-left", function(e){ Reveal.right(); })
.on("swipe-right", function(e){ Reveal.left(); })
.on("swipe-up", function(e){ Reveal.down(); })
.on("swipe-down", function(e){ Reveal.up(); })
.on("tap", function(e){ Reveal.next(); })
.on("zoom-out", function(e){ Reveal.toggleOverview(true); })
.on("zoom-in", function(e){ Reveal.toggleOverview(false); })
;
} );
head.js('https://hakim-static.s3.amazonaws.com/reveal-js/remotes.ne.min.js');
}
})(window);
doc/pub/Bayesian/html/._Bayesian-bs002.html
+218
View File
@@ -0,0 +1,218 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks">
<title>Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
('The Bayesian recipe', 2, None, '___sec4'),
("Bayes' theorem", 2, None, '___sec5'),
("The friends of Bayes' theorem", 2, None, '___sec6'),
('Inference With Parametric Models', 2, None, '___sec7'),
('Illustrative examples with python code', 2, None, '___sec8'),
('Example: Is this a fair coin?', 2, None, '___sec9'),
('A few words on different priors', 2, None, '___sec10'),
('Bayesian parameter estimation (single parameter)',
2,
None,
'___sec11'),
('Example: Measured flux from a star', 3, None, '___sec12'),
('Simple Photon Counts: Frequentist Approach',
3,
None,
'___sec13'),
('Simple Photon Counts: Bayesian Approach', 3, None, '___sec14'),
('A note about priors', 3, None, '___sec15'),
('Simple Photon Counts: Bayesian approach in practice',
3,
None,
'___sec16'),
('Best estimates and confidence intervals', 3, None, '___sec17'),
('Simple Photon Counts: Best estimates and confidence intervals',
3,
None,
'___sec18'),
('Bayesian parameter estimation (multiple parameters, '
'covariance)',
2,
None,
'___sec19'),
('Bayesian model selection', 2, None, '___sec20')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Bayesian-bs.html">Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>Why Bayesian Statistics?</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec1" style="font-size: 80%;"><b>Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs003.html#___sec2" style="font-size: 80%;"><b>Statistical Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs004.html#___sec3" style="font-size: 80%;"><b>Some history</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs010.html#___sec4" style="font-size: 80%;"><b>The Bayesian recipe</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs011.html#___sec5" style="font-size: 80%;"><b>Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs013.html#___sec6" style="font-size: 80%;"><b>The friends of Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs014.html#___sec7" style="font-size: 80%;"><b>Inference With Parametric Models</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs016.html#___sec8" style="font-size: 80%;"><b>Illustrative examples with python code</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs017.html#___sec9" style="font-size: 80%;"><b>Example: Is this a fair coin?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs018.html#___sec10" style="font-size: 80%;"><b>A few words on different priors</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs019.html#___sec11" style="font-size: 80%;"><b>Bayesian parameter estimation (single parameter)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec12" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Example: Measured flux from a star</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec13" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Frequentist Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec14" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec15" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;A note about priors</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec16" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian approach in practice</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec17" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec18" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs021.html#___sec19" style="font-size: 80%;"><b>Bayesian parameter estimation (multiple parameters, covariance)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs022.html#___sec20" style="font-size: 80%;"><b>Bayesian model selection</b></a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0002"></a>
<!-- !split -->
<h2 id="___sec1" class="anchor">Inference </h2>
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
<dl>
<dt>Inference:<dd>
&quot;the act of passing from one proposition, statement or judgment considered as true to another whose truth is believed to follow from that of the former&quot; (Webster) <br />
Do premises \( A, B, \ldots \to \) hypothesis, \( H \)?
<dt>Deductive inference:<dd>
Premises allow definite determination of truth/falsity of H (syllogisms, symbolic logic, Boolean algebra) <br />
\( B(H|A,B,...) = 0 \) or \( 1 \)
<dt>Inductive inference:<dd>
Premises bear on truth/falsity of H, but don&#8217;t allow its definite determination (weak syllogisms, analogies)<br />
\( A, B, C, D \) share properties \( x, y, z \); \( E \) has properties \( x, y \)<br />
\( \to \) $E$ probably has property \( z \).
</dl>
</div>
</div>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Bayesian-bs001.html">&laquo;</a></li>
<li><a href="._Bayesian-bs000.html">1</a></li>
<li><a href="._Bayesian-bs001.html">2</a></li>
<li class="active"><a href="._Bayesian-bs002.html">3</a></li>
<li><a href="._Bayesian-bs003.html">4</a></li>
<li><a href="._Bayesian-bs004.html">5</a></li>
<li><a href="._Bayesian-bs005.html">6</a></li>
<li><a href="._Bayesian-bs006.html">7</a></li>
<li><a href="._Bayesian-bs007.html">8</a></li>
<li><a href="._Bayesian-bs008.html">9</a></li>
<li><a href="._Bayesian-bs009.html">10</a></li>
<li><a href="._Bayesian-bs010.html">11</a></li>
<li><a href="._Bayesian-bs011.html">12</a></li>
<li><a href="">...</a></li>
<li><a href="._Bayesian-bs022.html">23</a></li>
<li><a href="._Bayesian-bs003.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/Bayesian/html/._Bayesian-bs003.html
+213
View File
@@ -0,0 +1,213 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks">
<title>Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
('The Bayesian recipe', 2, None, '___sec4'),
("Bayes' theorem", 2, None, '___sec5'),
("The friends of Bayes' theorem", 2, None, '___sec6'),
('Inference With Parametric Models', 2, None, '___sec7'),
('Illustrative examples with python code', 2, None, '___sec8'),
('Example: Is this a fair coin?', 2, None, '___sec9'),
('A few words on different priors', 2, None, '___sec10'),
('Bayesian parameter estimation (single parameter)',
2,
None,
'___sec11'),
('Example: Measured flux from a star', 3, None, '___sec12'),
('Simple Photon Counts: Frequentist Approach',
3,
None,
'___sec13'),
('Simple Photon Counts: Bayesian Approach', 3, None, '___sec14'),
('A note about priors', 3, None, '___sec15'),
('Simple Photon Counts: Bayesian approach in practice',
3,
None,
'___sec16'),
('Best estimates and confidence intervals', 3, None, '___sec17'),
('Simple Photon Counts: Best estimates and confidence intervals',
3,
None,
'___sec18'),
('Bayesian parameter estimation (multiple parameters, '
'covariance)',
2,
None,
'___sec19'),
('Bayesian model selection', 2, None, '___sec20')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Bayesian-bs.html">Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>Why Bayesian Statistics?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs002.html#___sec1" style="font-size: 80%;"><b>Inference</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec2" style="font-size: 80%;"><b>Statistical Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs004.html#___sec3" style="font-size: 80%;"><b>Some history</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs010.html#___sec4" style="font-size: 80%;"><b>The Bayesian recipe</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs011.html#___sec5" style="font-size: 80%;"><b>Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs013.html#___sec6" style="font-size: 80%;"><b>The friends of Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs014.html#___sec7" style="font-size: 80%;"><b>Inference With Parametric Models</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs016.html#___sec8" style="font-size: 80%;"><b>Illustrative examples with python code</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs017.html#___sec9" style="font-size: 80%;"><b>Example: Is this a fair coin?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs018.html#___sec10" style="font-size: 80%;"><b>A few words on different priors</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs019.html#___sec11" style="font-size: 80%;"><b>Bayesian parameter estimation (single parameter)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec12" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Example: Measured flux from a star</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec13" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Frequentist Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec14" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec15" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;A note about priors</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec16" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian approach in practice</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec17" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec18" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs021.html#___sec19" style="font-size: 80%;"><b>Bayesian parameter estimation (multiple parameters, covariance)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs022.html#___sec20" style="font-size: 80%;"><b>Bayesian model selection</b></a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0003"></a>
<!-- !split -->
<h2 id="___sec2" class="anchor">Statistical Inference </h2>
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
<ul>
<li> Quantify the strength of inductive inferences from facts, in the form of data (\( D \)), and other premises, e.g. models, to hypotheses about the phenomena producing the data.</li>
<li> Quantify via probabilities, or averages calculated using probabilities. Frequentists (\( \mathcal{F} \)) and Bayesians (\( \mathcal{B} \)) use probabilities very differently for this.</li>
<li> To the pioneers such as Bernoulli, Bayes and Laplace, a probability represented a <em>degree-of-belief</em> or plausability: how much they thought that something as true based on the evidence at hand. This is the Bayesian approach.</li>
<li> To the 19th century scholars, this seemed too vague and subjective. They redefined probability as the <em>long run relative frequency</em> with which an event occurred, given (infinitely) many repeated (experimental) trials.</li>
</ul>
</div>
</div>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Bayesian-bs002.html">&laquo;</a></li>
<li><a href="._Bayesian-bs000.html">1</a></li>
<li><a href="._Bayesian-bs001.html">2</a></li>
<li><a href="._Bayesian-bs002.html">3</a></li>
<li class="active"><a href="._Bayesian-bs003.html">4</a></li>
<li><a href="._Bayesian-bs004.html">5</a></li>
<li><a href="._Bayesian-bs005.html">6</a></li>
<li><a href="._Bayesian-bs006.html">7</a></li>
<li><a href="._Bayesian-bs007.html">8</a></li>
<li><a href="._Bayesian-bs008.html">9</a></li>
<li><a href="._Bayesian-bs009.html">10</a></li>
<li><a href="._Bayesian-bs010.html">11</a></li>
<li><a href="._Bayesian-bs011.html">12</a></li>
<li><a href="._Bayesian-bs012.html">13</a></li>
<li><a href="">...</a></li>
<li><a href="._Bayesian-bs022.html">23</a></li>
<li><a href="._Bayesian-bs004.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/Bayesian/html/._Bayesian-bs004.html
+210
View File
@@ -0,0 +1,210 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks">
<title>Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
('The Bayesian recipe', 2, None, '___sec4'),
("Bayes' theorem", 2, None, '___sec5'),
("The friends of Bayes' theorem", 2, None, '___sec6'),
('Inference With Parametric Models', 2, None, '___sec7'),
('Illustrative examples with python code', 2, None, '___sec8'),
('Example: Is this a fair coin?', 2, None, '___sec9'),
('A few words on different priors', 2, None, '___sec10'),
('Bayesian parameter estimation (single parameter)',
2,
None,
'___sec11'),
('Example: Measured flux from a star', 3, None, '___sec12'),
('Simple Photon Counts: Frequentist Approach',
3,
None,
'___sec13'),
('Simple Photon Counts: Bayesian Approach', 3, None, '___sec14'),
('A note about priors', 3, None, '___sec15'),
('Simple Photon Counts: Bayesian approach in practice',
3,
None,
'___sec16'),
('Best estimates and confidence intervals', 3, None, '___sec17'),
('Simple Photon Counts: Best estimates and confidence intervals',
3,
None,
'___sec18'),
('Bayesian parameter estimation (multiple parameters, '
'covariance)',
2,
None,
'___sec19'),
('Bayesian model selection', 2, None, '___sec20')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Bayesian-bs.html">Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>Why Bayesian Statistics?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs002.html#___sec1" style="font-size: 80%;"><b>Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs003.html#___sec2" style="font-size: 80%;"><b>Statistical Inference</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec3" style="font-size: 80%;"><b>Some history</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs010.html#___sec4" style="font-size: 80%;"><b>The Bayesian recipe</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs011.html#___sec5" style="font-size: 80%;"><b>Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs013.html#___sec6" style="font-size: 80%;"><b>The friends of Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs014.html#___sec7" style="font-size: 80%;"><b>Inference With Parametric Models</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs016.html#___sec8" style="font-size: 80%;"><b>Illustrative examples with python code</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs017.html#___sec9" style="font-size: 80%;"><b>Example: Is this a fair coin?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs018.html#___sec10" style="font-size: 80%;"><b>A few words on different priors</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs019.html#___sec11" style="font-size: 80%;"><b>Bayesian parameter estimation (single parameter)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec12" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Example: Measured flux from a star</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec13" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Frequentist Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec14" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec15" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;A note about priors</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec16" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian approach in practice</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec17" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec18" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs021.html#___sec19" style="font-size: 80%;"><b>Bayesian parameter estimation (multiple parameters, covariance)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs022.html#___sec20" style="font-size: 80%;"><b>Bayesian model selection</b></a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0004"></a>
<!-- !split -->
<h2 id="___sec3" class="anchor">Some history </h2>
Adapted from D.S. Sivia <button type="button" class="btn btn-primary btn-xs" rel="tooltip" data-placement="top" title="Sivia, Devinderjit, and John Skilling. Data Analysis : A Bayesian Tutorial, OUP Oxford, 2006"><a href="#def_footnote_1" id="link_footnote_1" style="color: white">1</a></button>:
<p id="def_footnote_1"><a href="#link_footnote_1"><b>1:</b></a> Sivia, Devinderjit, and John Skilling. Data Analysis : A Bayesian Tutorial, OUP Oxford, 2006</p>
<p>
<blockquote>
Although the frequency definition appears to be more objective, its range of validity is also far more limited. For example, Laplace used (his) probability theory to estimate the mass of Saturn, given orbital data that were available to him from various astronomical observatories. In essence, he computed the posterior pdf for the mass M , given the data and all the relevant background information I (such as a knowledge of the laws of classical mechanics): prob(M|{data},I); this is shown schematically in the figure [Fig. 1.2].
</blockquote>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Bayesian-bs003.html">&laquo;</a></li>
<li><a href="._Bayesian-bs000.html">1</a></li>
<li><a href="._Bayesian-bs001.html">2</a></li>
<li><a href="._Bayesian-bs002.html">3</a></li>
<li><a href="._Bayesian-bs003.html">4</a></li>
<li class="active"><a href="._Bayesian-bs004.html">5</a></li>
<li><a href="._Bayesian-bs005.html">6</a></li>
<li><a href="._Bayesian-bs006.html">7</a></li>
<li><a href="._Bayesian-bs007.html">8</a></li>
<li><a href="._Bayesian-bs008.html">9</a></li>
<li><a href="._Bayesian-bs009.html">10</a></li>
<li><a href="._Bayesian-bs010.html">11</a></li>
<li><a href="._Bayesian-bs011.html">12</a></li>
<li><a href="._Bayesian-bs012.html">13</a></li>
<li><a href="._Bayesian-bs013.html">14</a></li>
<li><a href="">...</a></li>
<li><a href="._Bayesian-bs022.html">23</a></li>
<li><a href="._Bayesian-bs005.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/Bayesian/html/._Bayesian-bs005.html
+201
View File
@@ -0,0 +1,201 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks">
<title>Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
('The Bayesian recipe', 2, None, '___sec4'),
("Bayes' theorem", 2, None, '___sec5'),
("The friends of Bayes' theorem", 2, None, '___sec6'),
('Inference With Parametric Models', 2, None, '___sec7'),
('Illustrative examples with python code', 2, None, '___sec8'),
('Example: Is this a fair coin?', 2, None, '___sec9'),
('A few words on different priors', 2, None, '___sec10'),
('Bayesian parameter estimation (single parameter)',
2,
None,
'___sec11'),
('Example: Measured flux from a star', 3, None, '___sec12'),
('Simple Photon Counts: Frequentist Approach',
3,
None,
'___sec13'),
('Simple Photon Counts: Bayesian Approach', 3, None, '___sec14'),
('A note about priors', 3, None, '___sec15'),
('Simple Photon Counts: Bayesian approach in practice',
3,
None,
'___sec16'),
('Best estimates and confidence intervals', 3, None, '___sec17'),
('Simple Photon Counts: Best estimates and confidence intervals',
3,
None,
'___sec18'),
('Bayesian parameter estimation (multiple parameters, '
'covariance)',
2,
None,
'___sec19'),
('Bayesian model selection', 2, None, '___sec20')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Bayesian-bs.html">Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>Why Bayesian Statistics?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs002.html#___sec1" style="font-size: 80%;"><b>Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs003.html#___sec2" style="font-size: 80%;"><b>Statistical Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs004.html#___sec3" style="font-size: 80%;"><b>Some history</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs010.html#___sec4" style="font-size: 80%;"><b>The Bayesian recipe</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs011.html#___sec5" style="font-size: 80%;"><b>Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs013.html#___sec6" style="font-size: 80%;"><b>The friends of Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs014.html#___sec7" style="font-size: 80%;"><b>Inference With Parametric Models</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs016.html#___sec8" style="font-size: 80%;"><b>Illustrative examples with python code</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs017.html#___sec9" style="font-size: 80%;"><b>Example: Is this a fair coin?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs018.html#___sec10" style="font-size: 80%;"><b>A few words on different priors</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs019.html#___sec11" style="font-size: 80%;"><b>Bayesian parameter estimation (single parameter)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec12" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Example: Measured flux from a star</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec13" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Frequentist Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec14" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec15" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;A note about priors</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec16" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian approach in practice</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec17" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec18" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs021.html#___sec19" style="font-size: 80%;"><b>Bayesian parameter estimation (multiple parameters, covariance)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs022.html#___sec20" style="font-size: 80%;"><b>Bayesian model selection</b></a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0005"></a>
<!-- !split -->
<br /><br /><center><p><img src="fig/sivia_fig_1_2.png" align="bottom" width=700></p></center><br /><br />
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Bayesian-bs004.html">&laquo;</a></li>
<li><a href="._Bayesian-bs000.html">1</a></li>
<li><a href="._Bayesian-bs001.html">2</a></li>
<li><a href="._Bayesian-bs002.html">3</a></li>
<li><a href="._Bayesian-bs003.html">4</a></li>
<li><a href="._Bayesian-bs004.html">5</a></li>
<li class="active"><a href="._Bayesian-bs005.html">6</a></li>
<li><a href="._Bayesian-bs006.html">7</a></li>
<li><a href="._Bayesian-bs007.html">8</a></li>
<li><a href="._Bayesian-bs008.html">9</a></li>
<li><a href="._Bayesian-bs009.html">10</a></li>
<li><a href="._Bayesian-bs010.html">11</a></li>
<li><a href="._Bayesian-bs011.html">12</a></li>
<li><a href="._Bayesian-bs012.html">13</a></li>
<li><a href="._Bayesian-bs013.html">14</a></li>
<li><a href="._Bayesian-bs014.html">15</a></li>
<li><a href="">...</a></li>
<li><a href="._Bayesian-bs022.html">23</a></li>
<li><a href="._Bayesian-bs006.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/Bayesian/html/._Bayesian-bs006.html
+205
View File
@@ -0,0 +1,205 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks">
<title>Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
('The Bayesian recipe', 2, None, '___sec4'),
("Bayes' theorem", 2, None, '___sec5'),
("The friends of Bayes' theorem", 2, None, '___sec6'),
('Inference With Parametric Models', 2, None, '___sec7'),
('Illustrative examples with python code', 2, None, '___sec8'),
('Example: Is this a fair coin?', 2, None, '___sec9'),
('A few words on different priors', 2, None, '___sec10'),
('Bayesian parameter estimation (single parameter)',
2,
None,
'___sec11'),
('Example: Measured flux from a star', 3, None, '___sec12'),
('Simple Photon Counts: Frequentist Approach',
3,
None,
'___sec13'),
('Simple Photon Counts: Bayesian Approach', 3, None, '___sec14'),
('A note about priors', 3, None, '___sec15'),
('Simple Photon Counts: Bayesian approach in practice',
3,
None,
'___sec16'),
('Best estimates and confidence intervals', 3, None, '___sec17'),
('Simple Photon Counts: Best estimates and confidence intervals',
3,
None,
'___sec18'),
('Bayesian parameter estimation (multiple parameters, '
'covariance)',
2,
None,
'___sec19'),
('Bayesian model selection', 2, None, '___sec20')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Bayesian-bs.html">Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>Why Bayesian Statistics?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs002.html#___sec1" style="font-size: 80%;"><b>Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs003.html#___sec2" style="font-size: 80%;"><b>Statistical Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs004.html#___sec3" style="font-size: 80%;"><b>Some history</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs010.html#___sec4" style="font-size: 80%;"><b>The Bayesian recipe</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs011.html#___sec5" style="font-size: 80%;"><b>Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs013.html#___sec6" style="font-size: 80%;"><b>The friends of Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs014.html#___sec7" style="font-size: 80%;"><b>Inference With Parametric Models</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs016.html#___sec8" style="font-size: 80%;"><b>Illustrative examples with python code</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs017.html#___sec9" style="font-size: 80%;"><b>Example: Is this a fair coin?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs018.html#___sec10" style="font-size: 80%;"><b>A few words on different priors</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs019.html#___sec11" style="font-size: 80%;"><b>Bayesian parameter estimation (single parameter)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec12" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Example: Measured flux from a star</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec13" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Frequentist Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec14" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec15" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;A note about priors</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec16" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian approach in practice</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec17" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec18" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs021.html#___sec19" style="font-size: 80%;"><b>Bayesian parameter estimation (multiple parameters, covariance)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs022.html#___sec20" style="font-size: 80%;"><b>Bayesian model selection</b></a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0006"></a>
<!-- !split -->
<blockquote>
To Laplace, the (shaded) area under the posterior pdf curve between \( m_1 \) and \( m_2 \) was a measure of how much he believed that the mass of Saturn lay in the range \( m_1 \le M \le m_2 \). As such, the position of the maximum of the posterior pdf represents a best estimate of the mass; its width, or spread, about this optimal value gives an indication of the uncertainty in the estimate. Laplace stated that: &#8216; . . . it is a bet of 11,000 to 1 that the error of this result is not 1/100th of its value.&#8217; He would have won the bet, as another 150 years&#8217; accumulation of data has changed the estimate by only 0.63%!
</blockquote>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Bayesian-bs005.html">&laquo;</a></li>
<li><a href="._Bayesian-bs000.html">1</a></li>
<li><a href="._Bayesian-bs001.html">2</a></li>
<li><a href="._Bayesian-bs002.html">3</a></li>
<li><a href="._Bayesian-bs003.html">4</a></li>
<li><a href="._Bayesian-bs004.html">5</a></li>
<li><a href="._Bayesian-bs005.html">6</a></li>
<li class="active"><a href="._Bayesian-bs006.html">7</a></li>
<li><a href="._Bayesian-bs007.html">8</a></li>
<li><a href="._Bayesian-bs008.html">9</a></li>
<li><a href="._Bayesian-bs009.html">10</a></li>
<li><a href="._Bayesian-bs010.html">11</a></li>
<li><a href="._Bayesian-bs011.html">12</a></li>
<li><a href="._Bayesian-bs012.html">13</a></li>
<li><a href="._Bayesian-bs013.html">14</a></li>
<li><a href="._Bayesian-bs014.html">15</a></li>
<li><a href="._Bayesian-bs015.html">16</a></li>
<li><a href="">...</a></li>
<li><a href="._Bayesian-bs022.html">23</a></li>
<li><a href="._Bayesian-bs007.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/Bayesian/html/._Bayesian-bs007.html
+209
View File
@@ -0,0 +1,209 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks">
<title>Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
('The Bayesian recipe', 2, None, '___sec4'),
("Bayes' theorem", 2, None, '___sec5'),
("The friends of Bayes' theorem", 2, None, '___sec6'),
('Inference With Parametric Models', 2, None, '___sec7'),
('Illustrative examples with python code', 2, None, '___sec8'),
('Example: Is this a fair coin?', 2, None, '___sec9'),
('A few words on different priors', 2, None, '___sec10'),
('Bayesian parameter estimation (single parameter)',
2,
None,
'___sec11'),
('Example: Measured flux from a star', 3, None, '___sec12'),
('Simple Photon Counts: Frequentist Approach',
3,
None,
'___sec13'),
('Simple Photon Counts: Bayesian Approach', 3, None, '___sec14'),
('A note about priors', 3, None, '___sec15'),
('Simple Photon Counts: Bayesian approach in practice',
3,
None,
'___sec16'),
('Best estimates and confidence intervals', 3, None, '___sec17'),
('Simple Photon Counts: Best estimates and confidence intervals',
3,
None,
'___sec18'),
('Bayesian parameter estimation (multiple parameters, '
'covariance)',
2,
None,
'___sec19'),
('Bayesian model selection', 2, None, '___sec20')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Bayesian-bs.html">Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>Why Bayesian Statistics?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs002.html#___sec1" style="font-size: 80%;"><b>Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs003.html#___sec2" style="font-size: 80%;"><b>Statistical Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs004.html#___sec3" style="font-size: 80%;"><b>Some history</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs010.html#___sec4" style="font-size: 80%;"><b>The Bayesian recipe</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs011.html#___sec5" style="font-size: 80%;"><b>Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs013.html#___sec6" style="font-size: 80%;"><b>The friends of Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs014.html#___sec7" style="font-size: 80%;"><b>Inference With Parametric Models</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs016.html#___sec8" style="font-size: 80%;"><b>Illustrative examples with python code</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs017.html#___sec9" style="font-size: 80%;"><b>Example: Is this a fair coin?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs018.html#___sec10" style="font-size: 80%;"><b>A few words on different priors</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs019.html#___sec11" style="font-size: 80%;"><b>Bayesian parameter estimation (single parameter)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec12" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Example: Measured flux from a star</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec13" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Frequentist Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec14" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec15" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;A note about priors</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec16" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian approach in practice</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec17" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec18" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs021.html#___sec19" style="font-size: 80%;"><b>Bayesian parameter estimation (multiple parameters, covariance)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs022.html#___sec20" style="font-size: 80%;"><b>Bayesian model selection</b></a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0007"></a>
<!-- !split -->
<blockquote>
According to the frequency definition, however, we are not permitted to use probability theory to tackle this problem. This is because the mass of Saturn is a constant and not a random variable; therefore, it has no frequency distribution and so probability theory cannot be used.
<p>
If the pdf [of Fig. 1.2] had to be interpreted in terms of the frequency definition, we would have to imagine a large ensemble of universes in which everything remains constant apart from the mass of Saturn.
</blockquote>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Bayesian-bs006.html">&laquo;</a></li>
<li><a href="._Bayesian-bs000.html">1</a></li>
<li><a href="._Bayesian-bs001.html">2</a></li>
<li><a href="._Bayesian-bs002.html">3</a></li>
<li><a href="._Bayesian-bs003.html">4</a></li>
<li><a href="._Bayesian-bs004.html">5</a></li>
<li><a href="._Bayesian-bs005.html">6</a></li>
<li><a href="._Bayesian-bs006.html">7</a></li>
<li class="active"><a href="._Bayesian-bs007.html">8</a></li>
<li><a href="._Bayesian-bs008.html">9</a></li>
<li><a href="._Bayesian-bs009.html">10</a></li>
<li><a href="._Bayesian-bs010.html">11</a></li>
<li><a href="._Bayesian-bs011.html">12</a></li>
<li><a href="._Bayesian-bs012.html">13</a></li>
<li><a href="._Bayesian-bs013.html">14</a></li>
<li><a href="._Bayesian-bs014.html">15</a></li>
<li><a href="._Bayesian-bs015.html">16</a></li>
<li><a href="._Bayesian-bs016.html">17</a></li>
<li><a href="">...</a></li>
<li><a href="._Bayesian-bs022.html">23</a></li>
<li><a href="._Bayesian-bs008.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/Bayesian/html/._Bayesian-bs008.html
+208
View File
@@ -0,0 +1,208 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks">
<title>Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
('The Bayesian recipe', 2, None, '___sec4'),
("Bayes' theorem", 2, None, '___sec5'),
("The friends of Bayes' theorem", 2, None, '___sec6'),
('Inference With Parametric Models', 2, None, '___sec7'),
('Illustrative examples with python code', 2, None, '___sec8'),
('Example: Is this a fair coin?', 2, None, '___sec9'),
('A few words on different priors', 2, None, '___sec10'),
('Bayesian parameter estimation (single parameter)',
2,
None,
'___sec11'),
('Example: Measured flux from a star', 3, None, '___sec12'),
('Simple Photon Counts: Frequentist Approach',
3,
None,
'___sec13'),
('Simple Photon Counts: Bayesian Approach', 3, None, '___sec14'),
('A note about priors', 3, None, '___sec15'),
('Simple Photon Counts: Bayesian approach in practice',
3,
None,
'___sec16'),
('Best estimates and confidence intervals', 3, None, '___sec17'),
('Simple Photon Counts: Best estimates and confidence intervals',
3,
None,
'___sec18'),
('Bayesian parameter estimation (multiple parameters, '
'covariance)',
2,
None,
'___sec19'),
('Bayesian model selection', 2, None, '___sec20')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Bayesian-bs.html">Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>Why Bayesian Statistics?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs002.html#___sec1" style="font-size: 80%;"><b>Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs003.html#___sec2" style="font-size: 80%;"><b>Statistical Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs004.html#___sec3" style="font-size: 80%;"><b>Some history</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs010.html#___sec4" style="font-size: 80%;"><b>The Bayesian recipe</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs011.html#___sec5" style="font-size: 80%;"><b>Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs013.html#___sec6" style="font-size: 80%;"><b>The friends of Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs014.html#___sec7" style="font-size: 80%;"><b>Inference With Parametric Models</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs016.html#___sec8" style="font-size: 80%;"><b>Illustrative examples with python code</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs017.html#___sec9" style="font-size: 80%;"><b>Example: Is this a fair coin?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs018.html#___sec10" style="font-size: 80%;"><b>A few words on different priors</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs019.html#___sec11" style="font-size: 80%;"><b>Bayesian parameter estimation (single parameter)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec12" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Example: Measured flux from a star</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec13" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Frequentist Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec14" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec15" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;A note about priors</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec16" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian approach in practice</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec17" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec18" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs021.html#___sec19" style="font-size: 80%;"><b>Bayesian parameter estimation (multiple parameters, covariance)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs022.html#___sec20" style="font-size: 80%;"><b>Bayesian model selection</b></a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0008"></a>
<!-- !split -->
<blockquote>
As this scenario appears quite far-fetched, we might be inclined to think of [Fig. 1.2] in terms of the distribution of the measurements of the mass in many repetitions of the experiment. Although we are at liberty to think about a problem in any way that facilitates its solution, or our understanding of it, having to seek a frequency interpretation for every data analysis problem seems rather perverse.
For example, what do we mean by the &#8216;measurement of the mass&#8217; when the data consist of orbital periods? Besides, why should we have to think about many repetitions of an experiment that never happened? What we really want to do is to make the best inference of the mass given the (few) data that we actually have; this is precisely the Bayes and Laplace view of probability.
</blockquote>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Bayesian-bs007.html">&laquo;</a></li>
<li><a href="._Bayesian-bs000.html">1</a></li>
<li><a href="._Bayesian-bs001.html">2</a></li>
<li><a href="._Bayesian-bs002.html">3</a></li>
<li><a href="._Bayesian-bs003.html">4</a></li>
<li><a href="._Bayesian-bs004.html">5</a></li>
<li><a href="._Bayesian-bs005.html">6</a></li>
<li><a href="._Bayesian-bs006.html">7</a></li>
<li><a href="._Bayesian-bs007.html">8</a></li>
<li class="active"><a href="._Bayesian-bs008.html">9</a></li>
<li><a href="._Bayesian-bs009.html">10</a></li>
<li><a href="._Bayesian-bs010.html">11</a></li>
<li><a href="._Bayesian-bs011.html">12</a></li>
<li><a href="._Bayesian-bs012.html">13</a></li>
<li><a href="._Bayesian-bs013.html">14</a></li>
<li><a href="._Bayesian-bs014.html">15</a></li>
<li><a href="._Bayesian-bs015.html">16</a></li>
<li><a href="._Bayesian-bs016.html">17</a></li>
<li><a href="._Bayesian-bs017.html">18</a></li>
<li><a href="">...</a></li>
<li><a href="._Bayesian-bs022.html">23</a></li>
<li><a href="._Bayesian-bs009.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/Bayesian/html/._Bayesian-bs009.html
+208
View File
@@ -0,0 +1,208 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks">
<title>Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
('The Bayesian recipe', 2, None, '___sec4'),
("Bayes' theorem", 2, None, '___sec5'),
("The friends of Bayes' theorem", 2, None, '___sec6'),
('Inference With Parametric Models', 2, None, '___sec7'),
('Illustrative examples with python code', 2, None, '___sec8'),
('Example: Is this a fair coin?', 2, None, '___sec9'),
('A few words on different priors', 2, None, '___sec10'),
('Bayesian parameter estimation (single parameter)',
2,
None,
'___sec11'),
('Example: Measured flux from a star', 3, None, '___sec12'),
('Simple Photon Counts: Frequentist Approach',
3,
None,
'___sec13'),
('Simple Photon Counts: Bayesian Approach', 3, None, '___sec14'),
('A note about priors', 3, None, '___sec15'),
('Simple Photon Counts: Bayesian approach in practice',
3,
None,
'___sec16'),
('Best estimates and confidence intervals', 3, None, '___sec17'),
('Simple Photon Counts: Best estimates and confidence intervals',
3,
None,
'___sec18'),
('Bayesian parameter estimation (multiple parameters, '
'covariance)',
2,
None,
'___sec19'),
('Bayesian model selection', 2, None, '___sec20')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Bayesian-bs.html">Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>Why Bayesian Statistics?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs002.html#___sec1" style="font-size: 80%;"><b>Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs003.html#___sec2" style="font-size: 80%;"><b>Statistical Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs004.html#___sec3" style="font-size: 80%;"><b>Some history</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs010.html#___sec4" style="font-size: 80%;"><b>The Bayesian recipe</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs011.html#___sec5" style="font-size: 80%;"><b>Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs013.html#___sec6" style="font-size: 80%;"><b>The friends of Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs014.html#___sec7" style="font-size: 80%;"><b>Inference With Parametric Models</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs016.html#___sec8" style="font-size: 80%;"><b>Illustrative examples with python code</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs017.html#___sec9" style="font-size: 80%;"><b>Example: Is this a fair coin?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs018.html#___sec10" style="font-size: 80%;"><b>A few words on different priors</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs019.html#___sec11" style="font-size: 80%;"><b>Bayesian parameter estimation (single parameter)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec12" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Example: Measured flux from a star</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec13" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Frequentist Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec14" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec15" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;A note about priors</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec16" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian approach in practice</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec17" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec18" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs021.html#___sec19" style="font-size: 80%;"><b>Bayesian parameter estimation (multiple parameters, covariance)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs022.html#___sec20" style="font-size: 80%;"><b>Bayesian model selection</b></a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0009"></a>
<!-- !split -->
<blockquote>
Faced with the realization that the frequency definition of probability theory did not permit most real-life scientific problems to be addressed, a new subject was invented &#8212; statistics! To estimate the mass of Saturn, for example, one has to relate the mass to the data through some function called the statistic; since the data are subject to &#8216;random&#8217; noise, the statistic becomes the random variable to which the rules of probability the- ory can be applied. But now the question arises: How should we choose the statistic? The frequentist approach does not yield a natural way of doing this and has, therefore, led to the development of several alternative schools of orthodox or conventional statis- tics. The masters, such as Fisher, Neyman and Pearson, provided a variety of different principles, which has merely resulted in a plethora of tests and procedures without any clear underlying rationale. This lack of unifying principles is, perhaps, at the heart of the shortcomings of the cook-book approach to statistics that students are often taught even today.
</blockquote>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Bayesian-bs008.html">&laquo;</a></li>
<li><a href="._Bayesian-bs000.html">1</a></li>
<li><a href="._Bayesian-bs001.html">2</a></li>
<li><a href="._Bayesian-bs002.html">3</a></li>
<li><a href="._Bayesian-bs003.html">4</a></li>
<li><a href="._Bayesian-bs004.html">5</a></li>
<li><a href="._Bayesian-bs005.html">6</a></li>
<li><a href="._Bayesian-bs006.html">7</a></li>
<li><a href="._Bayesian-bs007.html">8</a></li>
<li><a href="._Bayesian-bs008.html">9</a></li>
<li class="active"><a href="._Bayesian-bs009.html">10</a></li>
<li><a href="._Bayesian-bs010.html">11</a></li>
<li><a href="._Bayesian-bs011.html">12</a></li>
<li><a href="._Bayesian-bs012.html">13</a></li>
<li><a href="._Bayesian-bs013.html">14</a></li>
<li><a href="._Bayesian-bs014.html">15</a></li>
<li><a href="._Bayesian-bs015.html">16</a></li>
<li><a href="._Bayesian-bs016.html">17</a></li>
<li><a href="._Bayesian-bs017.html">18</a></li>
<li><a href="._Bayesian-bs018.html">19</a></li>
<li><a href="">...</a></li>
<li><a href="._Bayesian-bs022.html">23</a></li>
<li><a href="._Bayesian-bs010.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/Bayesian/html/._Bayesian-bs010.html
+233
View File
@@ -0,0 +1,233 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks">
<title>Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
('The Bayesian recipe', 2, None, '___sec4'),
("Bayes' theorem", 2, None, '___sec5'),
("The friends of Bayes' theorem", 2, None, '___sec6'),
('Inference With Parametric Models', 2, None, '___sec7'),
('Illustrative examples with python code', 2, None, '___sec8'),
('Example: Is this a fair coin?', 2, None, '___sec9'),
('A few words on different priors', 2, None, '___sec10'),
('Bayesian parameter estimation (single parameter)',
2,
None,
'___sec11'),
('Example: Measured flux from a star', 3, None, '___sec12'),
('Simple Photon Counts: Frequentist Approach',
3,
None,
'___sec13'),
('Simple Photon Counts: Bayesian Approach', 3, None, '___sec14'),
('A note about priors', 3, None, '___sec15'),
('Simple Photon Counts: Bayesian approach in practice',
3,
None,
'___sec16'),
('Best estimates and confidence intervals', 3, None, '___sec17'),
('Simple Photon Counts: Best estimates and confidence intervals',
3,
None,
'___sec18'),
('Bayesian parameter estimation (multiple parameters, '
'covariance)',
2,
None,
'___sec19'),
('Bayesian model selection', 2, None, '___sec20')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Bayesian-bs.html">Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>Why Bayesian Statistics?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs002.html#___sec1" style="font-size: 80%;"><b>Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs003.html#___sec2" style="font-size: 80%;"><b>Statistical Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs004.html#___sec3" style="font-size: 80%;"><b>Some history</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec4" style="font-size: 80%;"><b>The Bayesian recipe</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs011.html#___sec5" style="font-size: 80%;"><b>Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs013.html#___sec6" style="font-size: 80%;"><b>The friends of Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs014.html#___sec7" style="font-size: 80%;"><b>Inference With Parametric Models</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs016.html#___sec8" style="font-size: 80%;"><b>Illustrative examples with python code</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs017.html#___sec9" style="font-size: 80%;"><b>Example: Is this a fair coin?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs018.html#___sec10" style="font-size: 80%;"><b>A few words on different priors</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs019.html#___sec11" style="font-size: 80%;"><b>Bayesian parameter estimation (single parameter)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec12" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Example: Measured flux from a star</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec13" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Frequentist Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec14" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec15" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;A note about priors</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec16" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian approach in practice</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec17" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec18" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs021.html#___sec19" style="font-size: 80%;"><b>Bayesian parameter estimation (multiple parameters, covariance)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs022.html#___sec20" style="font-size: 80%;"><b>Bayesian model selection</b></a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0010"></a>
<!-- !split -->
<h2 id="___sec4" class="anchor">The Bayesian recipe </h2>
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
Assess hypotheses by calculating their probabilities \( p(H_i | \ldots) \) conditional on known and/or presumed information using the rules of probability theory.
</div>
</div>
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
Probability Theory Axioms:
<dl>
<dt>Product (AND) rule :<dd>
\( p(A, B | I) = p(A|I) p(B|A, I) = p(B|I)p(A|B,I) \)<br />
Should read \( p(A,B|I) \) as the probability for propositions \( A \) AND \( B \) being true given that \( I \) is true.
<dt>Sum (OR) rule:<dd>
\( p(A + B | I) = p(A | I) + p(B | I) - p(A, B | I) \)<br />
\( p(A+B|I) \) is the probability that proposition \( A \) OR \( B \) is true given that \( I \) is true.
<dt>Normalization:<dd>
\( p(A|I) + p(\bar{A}|I) = 1 \)<br />
\( \bar{A} \) denotes the proposition that \( A \) is false.
</dl>
</div>
</div>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Bayesian-bs009.html">&laquo;</a></li>
<li><a href="._Bayesian-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._Bayesian-bs002.html">3</a></li>
<li><a href="._Bayesian-bs003.html">4</a></li>
<li><a href="._Bayesian-bs004.html">5</a></li>
<li><a href="._Bayesian-bs005.html">6</a></li>
<li><a href="._Bayesian-bs006.html">7</a></li>
<li><a href="._Bayesian-bs007.html">8</a></li>
<li><a href="._Bayesian-bs008.html">9</a></li>
<li><a href="._Bayesian-bs009.html">10</a></li>
<li class="active"><a href="._Bayesian-bs010.html">11</a></li>
<li><a href="._Bayesian-bs011.html">12</a></li>
<li><a href="._Bayesian-bs012.html">13</a></li>
<li><a href="._Bayesian-bs013.html">14</a></li>
<li><a href="._Bayesian-bs014.html">15</a></li>
<li><a href="._Bayesian-bs015.html">16</a></li>
<li><a href="._Bayesian-bs016.html">17</a></li>
<li><a href="._Bayesian-bs017.html">18</a></li>
<li><a href="._Bayesian-bs018.html">19</a></li>
<li><a href="._Bayesian-bs019.html">20</a></li>
<li><a href="">...</a></li>
<li><a href="._Bayesian-bs022.html">23</a></li>
<li><a href="._Bayesian-bs011.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/Bayesian/html/._Bayesian-bs011.html
+229
View File
@@ -0,0 +1,229 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks">
<title>Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
('The Bayesian recipe', 2, None, '___sec4'),
("Bayes' theorem", 2, None, '___sec5'),
("The friends of Bayes' theorem", 2, None, '___sec6'),
('Inference With Parametric Models', 2, None, '___sec7'),
('Illustrative examples with python code', 2, None, '___sec8'),
('Example: Is this a fair coin?', 2, None, '___sec9'),
('A few words on different priors', 2, None, '___sec10'),
('Bayesian parameter estimation (single parameter)',
2,
None,
'___sec11'),
('Example: Measured flux from a star', 3, None, '___sec12'),
('Simple Photon Counts: Frequentist Approach',
3,
None,
'___sec13'),
('Simple Photon Counts: Bayesian Approach', 3, None, '___sec14'),
('A note about priors', 3, None, '___sec15'),
('Simple Photon Counts: Bayesian approach in practice',
3,
None,
'___sec16'),
('Best estimates and confidence intervals', 3, None, '___sec17'),
('Simple Photon Counts: Best estimates and confidence intervals',
3,
None,
'___sec18'),
('Bayesian parameter estimation (multiple parameters, '
'covariance)',
2,
None,
'___sec19'),
('Bayesian model selection', 2, None, '___sec20')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Bayesian-bs.html">Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>Why Bayesian Statistics?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs002.html#___sec1" style="font-size: 80%;"><b>Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs003.html#___sec2" style="font-size: 80%;"><b>Statistical Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs004.html#___sec3" style="font-size: 80%;"><b>Some history</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs010.html#___sec4" style="font-size: 80%;"><b>The Bayesian recipe</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec5" style="font-size: 80%;"><b>Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs013.html#___sec6" style="font-size: 80%;"><b>The friends of Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs014.html#___sec7" style="font-size: 80%;"><b>Inference With Parametric Models</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs016.html#___sec8" style="font-size: 80%;"><b>Illustrative examples with python code</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs017.html#___sec9" style="font-size: 80%;"><b>Example: Is this a fair coin?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs018.html#___sec10" style="font-size: 80%;"><b>A few words on different priors</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs019.html#___sec11" style="font-size: 80%;"><b>Bayesian parameter estimation (single parameter)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec12" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Example: Measured flux from a star</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec13" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Frequentist Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec14" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec15" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;A note about priors</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec16" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian approach in practice</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec17" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec18" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs021.html#___sec19" style="font-size: 80%;"><b>Bayesian parameter estimation (multiple parameters, covariance)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs022.html#___sec20" style="font-size: 80%;"><b>Bayesian model selection</b></a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0011"></a>
<!-- !split -->
<h2 id="___sec5" class="anchor">Bayes' theorem </h2>
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
Bayes' theorem follows directly from the product rule
$$
$$
p(A|B,I) = \frac{p(B|A,I) p(A|I)}{p(B|I)}.
$$
$$
The importance of this property to data analysis becomes apparent if we replace \( A \) and \( B \) by hypothesis(\( H \)) and data(\( D \)):
$$
\begin{align}
p(H|D,I) &= \frac{p(D|H,I) p(H|I)}{p(D|I)}.
\tag{1}
\end{align}
$$
The power of Bayes&#8217; theorem lies in the fact that it relates the quantity of interest, the probability that the hypothesis is true given the data, to the term we have a better chance of being able to assign, the probability that we would have observed the measured data if the hypothesis was true.
</div>
</div>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Bayesian-bs010.html">&laquo;</a></li>
<li><a href="._Bayesian-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._Bayesian-bs003.html">4</a></li>
<li><a href="._Bayesian-bs004.html">5</a></li>
<li><a href="._Bayesian-bs005.html">6</a></li>
<li><a href="._Bayesian-bs006.html">7</a></li>
<li><a href="._Bayesian-bs007.html">8</a></li>
<li><a href="._Bayesian-bs008.html">9</a></li>
<li><a href="._Bayesian-bs009.html">10</a></li>
<li><a href="._Bayesian-bs010.html">11</a></li>
<li class="active"><a href="._Bayesian-bs011.html">12</a></li>
<li><a href="._Bayesian-bs012.html">13</a></li>
<li><a href="._Bayesian-bs013.html">14</a></li>
<li><a href="._Bayesian-bs014.html">15</a></li>
<li><a href="._Bayesian-bs015.html">16</a></li>
<li><a href="._Bayesian-bs016.html">17</a></li>
<li><a href="._Bayesian-bs017.html">18</a></li>
<li><a href="._Bayesian-bs018.html">19</a></li>
<li><a href="._Bayesian-bs019.html">20</a></li>
<li><a href="._Bayesian-bs020.html">21</a></li>
<li><a href="">...</a></li>
<li><a href="._Bayesian-bs022.html">23</a></li>
<li><a href="._Bayesian-bs012.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/Bayesian/html/._Bayesian-bs012.html
+221
View File
@@ -0,0 +1,221 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks">
<title>Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
('The Bayesian recipe', 2, None, '___sec4'),
("Bayes' theorem", 2, None, '___sec5'),
("The friends of Bayes' theorem", 2, None, '___sec6'),
('Inference With Parametric Models', 2, None, '___sec7'),
('Illustrative examples with python code', 2, None, '___sec8'),
('Example: Is this a fair coin?', 2, None, '___sec9'),
('A few words on different priors', 2, None, '___sec10'),
('Bayesian parameter estimation (single parameter)',
2,
None,
'___sec11'),
('Example: Measured flux from a star', 3, None, '___sec12'),
('Simple Photon Counts: Frequentist Approach',
3,
None,
'___sec13'),
('Simple Photon Counts: Bayesian Approach', 3, None, '___sec14'),
('A note about priors', 3, None, '___sec15'),
('Simple Photon Counts: Bayesian approach in practice',
3,
None,
'___sec16'),
('Best estimates and confidence intervals', 3, None, '___sec17'),
('Simple Photon Counts: Best estimates and confidence intervals',
3,
None,
'___sec18'),
('Bayesian parameter estimation (multiple parameters, '
'covariance)',
2,
None,
'___sec19'),
('Bayesian model selection', 2, None, '___sec20')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Bayesian-bs.html">Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>Why Bayesian Statistics?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs002.html#___sec1" style="font-size: 80%;"><b>Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs003.html#___sec2" style="font-size: 80%;"><b>Statistical Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs004.html#___sec3" style="font-size: 80%;"><b>Some history</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs010.html#___sec4" style="font-size: 80%;"><b>The Bayesian recipe</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs011.html#___sec5" style="font-size: 80%;"><b>Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs013.html#___sec6" style="font-size: 80%;"><b>The friends of Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs014.html#___sec7" style="font-size: 80%;"><b>Inference With Parametric Models</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs016.html#___sec8" style="font-size: 80%;"><b>Illustrative examples with python code</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs017.html#___sec9" style="font-size: 80%;"><b>Example: Is this a fair coin?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs018.html#___sec10" style="font-size: 80%;"><b>A few words on different priors</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs019.html#___sec11" style="font-size: 80%;"><b>Bayesian parameter estimation (single parameter)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec12" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Example: Measured flux from a star</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec13" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Frequentist Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec14" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec15" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;A note about priors</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec16" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian approach in practice</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec17" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec18" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs021.html#___sec19" style="font-size: 80%;"><b>Bayesian parameter estimation (multiple parameters, covariance)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs022.html#___sec20" style="font-size: 80%;"><b>Bayesian model selection</b></a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0012"></a>
<!-- !split -->
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
The various terms in Bayes&#8217; theorem have formal names.
<ul>
<li> The quantity on the far right, \( p(H|I) \), is called the <em>prior</em> probability; it represents our state of knowledge (or ignorance) about the truth of the hypothesis before we have analysed the current data.</li>
<li> This is modified by the experimental measurements through \( p(D|H,I) \), the <em>likelihood</em> function,</li>
<li> The denominator \( p(D|I) \) is called the <em>evidence</em>. It does not depend on the hypothesis and can be regarded as a normalization constant.</li>
<li> Together, these yield the <em>posterior</em> probability, \( p(H|D, I ) \), representing our state of knowledge about the truth of the hypothesis in the light of the data.</li>
</ul>
In a sense, Bayes&#8217; theorem encapsulates the process of learning.
</div>
</div>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Bayesian-bs011.html">&laquo;</a></li>
<li><a href="._Bayesian-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._Bayesian-bs004.html">5</a></li>
<li><a href="._Bayesian-bs005.html">6</a></li>
<li><a href="._Bayesian-bs006.html">7</a></li>
<li><a href="._Bayesian-bs007.html">8</a></li>
<li><a href="._Bayesian-bs008.html">9</a></li>
<li><a href="._Bayesian-bs009.html">10</a></li>
<li><a href="._Bayesian-bs010.html">11</a></li>
<li><a href="._Bayesian-bs011.html">12</a></li>
<li class="active"><a href="._Bayesian-bs012.html">13</a></li>
<li><a href="._Bayesian-bs013.html">14</a></li>
<li><a href="._Bayesian-bs014.html">15</a></li>
<li><a href="._Bayesian-bs015.html">16</a></li>
<li><a href="._Bayesian-bs016.html">17</a></li>
<li><a href="._Bayesian-bs017.html">18</a></li>
<li><a href="._Bayesian-bs018.html">19</a></li>
<li><a href="._Bayesian-bs019.html">20</a></li>
<li><a href="._Bayesian-bs020.html">21</a></li>
<li><a href="._Bayesian-bs021.html">22</a></li>
<li><a href="">...</a></li>
<li><a href="._Bayesian-bs022.html">23</a></li>
<li><a href="._Bayesian-bs013.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/Bayesian/html/._Bayesian-bs013.html
+228
View File
@@ -0,0 +1,228 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks">
<title>Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
('The Bayesian recipe', 2, None, '___sec4'),
("Bayes' theorem", 2, None, '___sec5'),
("The friends of Bayes' theorem", 2, None, '___sec6'),
('Inference With Parametric Models', 2, None, '___sec7'),
('Illustrative examples with python code', 2, None, '___sec8'),
('Example: Is this a fair coin?', 2, None, '___sec9'),
('A few words on different priors', 2, None, '___sec10'),
('Bayesian parameter estimation (single parameter)',
2,
None,
'___sec11'),
('Example: Measured flux from a star', 3, None, '___sec12'),
('Simple Photon Counts: Frequentist Approach',
3,
None,
'___sec13'),
('Simple Photon Counts: Bayesian Approach', 3, None, '___sec14'),
('A note about priors', 3, None, '___sec15'),
('Simple Photon Counts: Bayesian approach in practice',
3,
None,
'___sec16'),
('Best estimates and confidence intervals', 3, None, '___sec17'),
('Simple Photon Counts: Best estimates and confidence intervals',
3,
None,
'___sec18'),
('Bayesian parameter estimation (multiple parameters, '
'covariance)',
2,
None,
'___sec19'),
('Bayesian model selection', 2, None, '___sec20')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Bayesian-bs.html">Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>Why Bayesian Statistics?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs002.html#___sec1" style="font-size: 80%;"><b>Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs003.html#___sec2" style="font-size: 80%;"><b>Statistical Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs004.html#___sec3" style="font-size: 80%;"><b>Some history</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs010.html#___sec4" style="font-size: 80%;"><b>The Bayesian recipe</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs011.html#___sec5" style="font-size: 80%;"><b>Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec6" style="font-size: 80%;"><b>The friends of Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs014.html#___sec7" style="font-size: 80%;"><b>Inference With Parametric Models</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs016.html#___sec8" style="font-size: 80%;"><b>Illustrative examples with python code</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs017.html#___sec9" style="font-size: 80%;"><b>Example: Is this a fair coin?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs018.html#___sec10" style="font-size: 80%;"><b>A few words on different priors</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs019.html#___sec11" style="font-size: 80%;"><b>Bayesian parameter estimation (single parameter)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec12" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Example: Measured flux from a star</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec13" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Frequentist Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec14" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec15" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;A note about priors</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec16" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian approach in practice</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec17" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec18" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs021.html#___sec19" style="font-size: 80%;"><b>Bayesian parameter estimation (multiple parameters, covariance)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs022.html#___sec20" style="font-size: 80%;"><b>Bayesian model selection</b></a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0013"></a>
<!-- !split -->
<h2 id="___sec6" class="anchor">The friends of Bayes' theorem </h2>
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
<dl>
<dt>Normalization:<dd>
\( \sum_i p(H_i|\ldots) = 1 \).
<dt>Marginalization:<dd>
\( \sum_i p(A,H_i|I) = \sum_i p(H_i|A,I) p(A|I) = p(A|I) \).
<dt>Marginalization (continuum limit):<dd>
\( \int dx p(A,H(x)|I) = p(A|I) \).
</dl>
In the above, \( H_i \) is an exclusive and exhaustive list of hypotheses. For example,let&#8217;s imagine that there are five candidates in a presidential election; then \( H_1 \) could be the proposition that the first candidate will win, and so on. The probability that \( A \) is true, for example that unemployment will be lower in a year&#8217;s time (given all relevant information \( I \), but irrespective of whoever becomes president) is then given by \( \sum_i p(A,H_i|I) \).
<p>
In the continuum limit of propositions we must understand \( p(\ldots) \) as a pdf (probability density function).
<p>
Marginalization is a very powerful device in data analysis because it enables us to deal with nuisance parameters; that is, quantities which necessarily enter the analysis but are of no intrinsic interest. The unwanted background signal present in many experimental measurements are examples of nuisance parameters.
</div>
</div>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Bayesian-bs012.html">&laquo;</a></li>
<li><a href="._Bayesian-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._Bayesian-bs005.html">6</a></li>
<li><a href="._Bayesian-bs006.html">7</a></li>
<li><a href="._Bayesian-bs007.html">8</a></li>
<li><a href="._Bayesian-bs008.html">9</a></li>
<li><a href="._Bayesian-bs009.html">10</a></li>
<li><a href="._Bayesian-bs010.html">11</a></li>
<li><a href="._Bayesian-bs011.html">12</a></li>
<li><a href="._Bayesian-bs012.html">13</a></li>
<li class="active"><a href="._Bayesian-bs013.html">14</a></li>
<li><a href="._Bayesian-bs014.html">15</a></li>
<li><a href="._Bayesian-bs015.html">16</a></li>
<li><a href="._Bayesian-bs016.html">17</a></li>
<li><a href="._Bayesian-bs017.html">18</a></li>
<li><a href="._Bayesian-bs018.html">19</a></li>
<li><a href="._Bayesian-bs019.html">20</a></li>
<li><a href="._Bayesian-bs020.html">21</a></li>
<li><a href="._Bayesian-bs021.html">22</a></li>
<li><a href="._Bayesian-bs022.html">23</a></li>
<li><a href="._Bayesian-bs014.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/Bayesian/html/._Bayesian-bs014.html
+238
View File
@@ -0,0 +1,238 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks">
<title>Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
('The Bayesian recipe', 2, None, '___sec4'),
("Bayes' theorem", 2, None, '___sec5'),
("The friends of Bayes' theorem", 2, None, '___sec6'),
('Inference With Parametric Models', 2, None, '___sec7'),
('Illustrative examples with python code', 2, None, '___sec8'),
('Example: Is this a fair coin?', 2, None, '___sec9'),
('A few words on different priors', 2, None, '___sec10'),
('Bayesian parameter estimation (single parameter)',
2,
None,
'___sec11'),
('Example: Measured flux from a star', 3, None, '___sec12'),
('Simple Photon Counts: Frequentist Approach',
3,
None,
'___sec13'),
('Simple Photon Counts: Bayesian Approach', 3, None, '___sec14'),
('A note about priors', 3, None, '___sec15'),
('Simple Photon Counts: Bayesian approach in practice',
3,
None,
'___sec16'),
('Best estimates and confidence intervals', 3, None, '___sec17'),
('Simple Photon Counts: Best estimates and confidence intervals',
3,
None,
'___sec18'),
('Bayesian parameter estimation (multiple parameters, '
'covariance)',
2,
None,
'___sec19'),
('Bayesian model selection', 2, None, '___sec20')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Bayesian-bs.html">Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>Why Bayesian Statistics?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs002.html#___sec1" style="font-size: 80%;"><b>Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs003.html#___sec2" style="font-size: 80%;"><b>Statistical Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs004.html#___sec3" style="font-size: 80%;"><b>Some history</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs010.html#___sec4" style="font-size: 80%;"><b>The Bayesian recipe</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs011.html#___sec5" style="font-size: 80%;"><b>Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs013.html#___sec6" style="font-size: 80%;"><b>The friends of Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec7" style="font-size: 80%;"><b>Inference With Parametric Models</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs016.html#___sec8" style="font-size: 80%;"><b>Illustrative examples with python code</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs017.html#___sec9" style="font-size: 80%;"><b>Example: Is this a fair coin?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs018.html#___sec10" style="font-size: 80%;"><b>A few words on different priors</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs019.html#___sec11" style="font-size: 80%;"><b>Bayesian parameter estimation (single parameter)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec12" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Example: Measured flux from a star</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec13" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Frequentist Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec14" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec15" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;A note about priors</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec16" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian approach in practice</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec17" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec18" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs021.html#___sec19" style="font-size: 80%;"><b>Bayesian parameter estimation (multiple parameters, covariance)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs022.html#___sec20" style="font-size: 80%;"><b>Bayesian model selection</b></a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0014"></a>
<!-- !split -->
<h2 id="___sec7" class="anchor">Inference With Parametric Models </h2>
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
Inductive inference with parametric models is a very important tool in the natural sciences.
<ul>
<li> Consider \( N \) different models \( M_i \) (\( i = 1, \ldots, N \)), each with parameters \( \boldsymbol{\alpha}_i \). Each of them implies a sampling distribution (conditional predictive distribution for possible data)</li>
</ul>
$$
$$
p(D|\boldsymbol{\alpha}_i, M_i)
$$
$$
<ul>
<li> The \( \boldsymbol{\alpha}_i \) dependence when we fix attention on the actual, observed data (\( D_\mathrm{obs} \)) is the likelihood function:</li>
</ul>
$$
$$
\mathcal{L}_i (\boldsymbol{\alpha}_i) \equiv p(D_\mathrm{obs}|\boldsymbol{\alpha}_i, M_i)
$$
$$
<ul>
<li> We may be uncertain about \( i \) (model uncertainty),</li>
<li> or uncertain about \( \boldsymbol{\alpha}_i \) (parameter uncertainty).</li>
</ul>
</div>
</div>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Bayesian-bs013.html">&laquo;</a></li>
<li><a href="._Bayesian-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._Bayesian-bs006.html">7</a></li>
<li><a href="._Bayesian-bs007.html">8</a></li>
<li><a href="._Bayesian-bs008.html">9</a></li>
<li><a href="._Bayesian-bs009.html">10</a></li>
<li><a href="._Bayesian-bs010.html">11</a></li>
<li><a href="._Bayesian-bs011.html">12</a></li>
<li><a href="._Bayesian-bs012.html">13</a></li>
<li><a href="._Bayesian-bs013.html">14</a></li>
<li class="active"><a href="._Bayesian-bs014.html">15</a></li>
<li><a href="._Bayesian-bs015.html">16</a></li>
<li><a href="._Bayesian-bs016.html">17</a></li>
<li><a href="._Bayesian-bs017.html">18</a></li>
<li><a href="._Bayesian-bs018.html">19</a></li>
<li><a href="._Bayesian-bs019.html">20</a></li>
<li><a href="._Bayesian-bs020.html">21</a></li>
<li><a href="._Bayesian-bs021.html">22</a></li>
<li><a href="._Bayesian-bs022.html">23</a></li>
<li><a href="._Bayesian-bs015.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/Bayesian/html/._Bayesian-bs015.html
+222
View File
@@ -0,0 +1,222 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks">
<title>Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
('The Bayesian recipe', 2, None, '___sec4'),
("Bayes' theorem", 2, None, '___sec5'),
("The friends of Bayes' theorem", 2, None, '___sec6'),
('Inference With Parametric Models', 2, None, '___sec7'),
('Illustrative examples with python code', 2, None, '___sec8'),
('Example: Is this a fair coin?', 2, None, '___sec9'),
('A few words on different priors', 2, None, '___sec10'),
('Bayesian parameter estimation (single parameter)',
2,
None,
'___sec11'),
('Example: Measured flux from a star', 3, None, '___sec12'),
('Simple Photon Counts: Frequentist Approach',
3,
None,
'___sec13'),
('Simple Photon Counts: Bayesian Approach', 3, None, '___sec14'),
('A note about priors', 3, None, '___sec15'),
('Simple Photon Counts: Bayesian approach in practice',
3,
None,
'___sec16'),
('Best estimates and confidence intervals', 3, None, '___sec17'),
('Simple Photon Counts: Best estimates and confidence intervals',
3,
None,
'___sec18'),
('Bayesian parameter estimation (multiple parameters, '
'covariance)',
2,
None,
'___sec19'),
('Bayesian model selection', 2, None, '___sec20')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Bayesian-bs.html">Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>Why Bayesian Statistics?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs002.html#___sec1" style="font-size: 80%;"><b>Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs003.html#___sec2" style="font-size: 80%;"><b>Statistical Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs004.html#___sec3" style="font-size: 80%;"><b>Some history</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs010.html#___sec4" style="font-size: 80%;"><b>The Bayesian recipe</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs011.html#___sec5" style="font-size: 80%;"><b>Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs013.html#___sec6" style="font-size: 80%;"><b>The friends of Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs014.html#___sec7" style="font-size: 80%;"><b>Inference With Parametric Models</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs016.html#___sec8" style="font-size: 80%;"><b>Illustrative examples with python code</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs017.html#___sec9" style="font-size: 80%;"><b>Example: Is this a fair coin?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs018.html#___sec10" style="font-size: 80%;"><b>A few words on different priors</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs019.html#___sec11" style="font-size: 80%;"><b>Bayesian parameter estimation (single parameter)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec12" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Example: Measured flux from a star</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec13" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Frequentist Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec14" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec15" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;A note about priors</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec16" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian approach in practice</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec17" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec18" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs021.html#___sec19" style="font-size: 80%;"><b>Bayesian parameter estimation (multiple parameters, covariance)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs022.html#___sec20" style="font-size: 80%;"><b>Bayesian model selection</b></a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0015"></a>
<!-- !split -->
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
<dl>
<dt>Parameter Estimation:<dd>
Premise = choice of model (pick specific \( i \))<br />
\( \Rightarrow \) What can we say about \( \boldsymbol{\alpha}_i \)?
<dt>Model comparison:<dd>
Premise = \( \{M_i\} \)<br />
\( \Rightarrow \) What can we say about \( i \)?
<dt>Model adequacy:<dd>
Premise = \( M_1 \)<br />
\( \Rightarrow \) Is \( M_1 \) adequate?
<dt>Hybrid Uncertainty:<dd>
Models share some common params: \( \boldsymbol{\alpha}_1 = \{ \boldsymbol{\varphi}, \boldsymbol{\eta}_i\} \)<br />
\( \Rightarrow \) What can we say about \( \boldsymbol{\varphi} \)? (Systematic error is an example)
</dl>
</div>
</div>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Bayesian-bs014.html">&laquo;</a></li>
<li><a href="._Bayesian-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._Bayesian-bs007.html">8</a></li>
<li><a href="._Bayesian-bs008.html">9</a></li>
<li><a href="._Bayesian-bs009.html">10</a></li>
<li><a href="._Bayesian-bs010.html">11</a></li>
<li><a href="._Bayesian-bs011.html">12</a></li>
<li><a href="._Bayesian-bs012.html">13</a></li>
<li><a href="._Bayesian-bs013.html">14</a></li>
<li><a href="._Bayesian-bs014.html">15</a></li>
<li class="active"><a href="._Bayesian-bs015.html">16</a></li>
<li><a href="._Bayesian-bs016.html">17</a></li>
<li><a href="._Bayesian-bs017.html">18</a></li>
<li><a href="._Bayesian-bs018.html">19</a></li>
<li><a href="._Bayesian-bs019.html">20</a></li>
<li><a href="._Bayesian-bs020.html">21</a></li>
<li><a href="._Bayesian-bs021.html">22</a></li>
<li><a href="._Bayesian-bs022.html">23</a></li>
<li><a href="._Bayesian-bs016.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/Bayesian/html/._Bayesian-bs016.html
+216
View File
@@ -0,0 +1,216 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks">
<title>Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
('The Bayesian recipe', 2, None, '___sec4'),
("Bayes' theorem", 2, None, '___sec5'),
("The friends of Bayes' theorem", 2, None, '___sec6'),
('Inference With Parametric Models', 2, None, '___sec7'),
('Illustrative examples with python code', 2, None, '___sec8'),
('Example: Is this a fair coin?', 2, None, '___sec9'),
('A few words on different priors', 2, None, '___sec10'),
('Bayesian parameter estimation (single parameter)',
2,
None,
'___sec11'),
('Example: Measured flux from a star', 3, None, '___sec12'),
('Simple Photon Counts: Frequentist Approach',
3,
None,
'___sec13'),
('Simple Photon Counts: Bayesian Approach', 3, None, '___sec14'),
('A note about priors', 3, None, '___sec15'),
('Simple Photon Counts: Bayesian approach in practice',
3,
None,
'___sec16'),
('Best estimates and confidence intervals', 3, None, '___sec17'),
('Simple Photon Counts: Best estimates and confidence intervals',
3,
None,
'___sec18'),
('Bayesian parameter estimation (multiple parameters, '
'covariance)',
2,
None,
'___sec19'),
('Bayesian model selection', 2, None, '___sec20')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Bayesian-bs.html">Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>Why Bayesian Statistics?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs002.html#___sec1" style="font-size: 80%;"><b>Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs003.html#___sec2" style="font-size: 80%;"><b>Statistical Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs004.html#___sec3" style="font-size: 80%;"><b>Some history</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs010.html#___sec4" style="font-size: 80%;"><b>The Bayesian recipe</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs011.html#___sec5" style="font-size: 80%;"><b>Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs013.html#___sec6" style="font-size: 80%;"><b>The friends of Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs014.html#___sec7" style="font-size: 80%;"><b>Inference With Parametric Models</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec8" style="font-size: 80%;"><b>Illustrative examples with python code</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs017.html#___sec9" style="font-size: 80%;"><b>Example: Is this a fair coin?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs018.html#___sec10" style="font-size: 80%;"><b>A few words on different priors</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs019.html#___sec11" style="font-size: 80%;"><b>Bayesian parameter estimation (single parameter)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec12" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Example: Measured flux from a star</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec13" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Frequentist Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec14" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec15" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;A note about priors</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec16" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian approach in practice</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec17" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec18" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs021.html#___sec19" style="font-size: 80%;"><b>Bayesian parameter estimation (multiple parameters, covariance)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs022.html#___sec20" style="font-size: 80%;"><b>Bayesian model selection</b></a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0016"></a>
<!-- !split -->
<h2 id="___sec8" class="anchor">Illustrative examples with python code </h2>
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
<ul>
<li> Is this a fair coin? (analytical)</li>
<li> Flux from a star (single parameter, MCMC)</li>
<li> The lighthouse problem (two parameters, MCMC)</li>
<li> Linear fit with outliers (nuisance parameters)</li>
<li> ...</li>
</ul>
</div>
</div>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Bayesian-bs015.html">&laquo;</a></li>
<li><a href="._Bayesian-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._Bayesian-bs008.html">9</a></li>
<li><a href="._Bayesian-bs009.html">10</a></li>
<li><a href="._Bayesian-bs010.html">11</a></li>
<li><a href="._Bayesian-bs011.html">12</a></li>
<li><a href="._Bayesian-bs012.html">13</a></li>
<li><a href="._Bayesian-bs013.html">14</a></li>
<li><a href="._Bayesian-bs014.html">15</a></li>
<li><a href="._Bayesian-bs015.html">16</a></li>
<li class="active"><a href="._Bayesian-bs016.html">17</a></li>
<li><a href="._Bayesian-bs017.html">18</a></li>
<li><a href="._Bayesian-bs018.html">19</a></li>
<li><a href="._Bayesian-bs019.html">20</a></li>
<li><a href="._Bayesian-bs020.html">21</a></li>
<li><a href="._Bayesian-bs021.html">22</a></li>
<li><a href="._Bayesian-bs022.html">23</a></li>
<li><a href="._Bayesian-bs017.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/Bayesian/html/._Bayesian-bs017.html
+313
View File
@@ -0,0 +1,313 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks">
<title>Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
('The Bayesian recipe', 2, None, '___sec4'),
("Bayes' theorem", 2, None, '___sec5'),
("The friends of Bayes' theorem", 2, None, '___sec6'),
('Inference With Parametric Models', 2, None, '___sec7'),
('Illustrative examples with python code', 2, None, '___sec8'),
('Example: Is this a fair coin?', 2, None, '___sec9'),
('A few words on different priors', 2, None, '___sec10'),
('Bayesian parameter estimation (single parameter)',
2,
None,
'___sec11'),
('Example: Measured flux from a star', 3, None, '___sec12'),
('Simple Photon Counts: Frequentist Approach',
3,
None,
'___sec13'),
('Simple Photon Counts: Bayesian Approach', 3, None, '___sec14'),
('A note about priors', 3, None, '___sec15'),
('Simple Photon Counts: Bayesian approach in practice',
3,
None,
'___sec16'),
('Best estimates and confidence intervals', 3, None, '___sec17'),
('Simple Photon Counts: Best estimates and confidence intervals',
3,
None,
'___sec18'),
('Bayesian parameter estimation (multiple parameters, '
'covariance)',
2,
None,
'___sec19'),
('Bayesian model selection', 2, None, '___sec20')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Bayesian-bs.html">Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>Why Bayesian Statistics?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs002.html#___sec1" style="font-size: 80%;"><b>Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs003.html#___sec2" style="font-size: 80%;"><b>Statistical Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs004.html#___sec3" style="font-size: 80%;"><b>Some history</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs010.html#___sec4" style="font-size: 80%;"><b>The Bayesian recipe</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs011.html#___sec5" style="font-size: 80%;"><b>Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs013.html#___sec6" style="font-size: 80%;"><b>The friends of Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs014.html#___sec7" style="font-size: 80%;"><b>Inference With Parametric Models</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs016.html#___sec8" style="font-size: 80%;"><b>Illustrative examples with python code</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec9" style="font-size: 80%;"><b>Example: Is this a fair coin?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs018.html#___sec10" style="font-size: 80%;"><b>A few words on different priors</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs019.html#___sec11" style="font-size: 80%;"><b>Bayesian parameter estimation (single parameter)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec12" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Example: Measured flux from a star</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec13" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Frequentist Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec14" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec15" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;A note about priors</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec16" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian approach in practice</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec17" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec18" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs021.html#___sec19" style="font-size: 80%;"><b>Bayesian parameter estimation (multiple parameters, covariance)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs022.html#___sec20" style="font-size: 80%;"><b>Bayesian model selection</b></a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0017"></a>
<!-- !split -->
<h2 id="___sec9" class="anchor">Example: Is this a fair coin? </h2>
Let us begin with the analysis of data from a simple coin-tossing experiment.
Given that we had observed 6 heads in 8 flips, would you think it was a fair coin? By fair, we mean that we would be prepared to lay an even 1 : 1 bet on the outcome of a flip being a head or a tail. If we decide that the coin was fair, the question which follows naturally is how sure are we that this was so; if it was not fair, how unfair do we think it was? Furthermore, if we were to continue collecting data for this particular coin, observing the outcomes of additional flips, how would we update our belief on the fairness of the coin?
<p>
A sensible way of formulating this problem is to consider a large number of hypotheses about the range in which the bias-weighting of the coin might lie. If we denote the bias-weighting by \( H \), then \( H = 0 \) and \( H = 1 \) can represent a coin which produces a tail or a head on every flip, respectively. There is a continuum of possibilities for the value of H between these limits, with \( H = 0.5 \) indicating a fair coin. Our state of knowledge about the fairness, or the degree of unfairness, of the coin is then completely summarized by specifying how much we believe these various propositions to be true.
<p>
Let us perform a computer simulation of a coin-tossing experiment. This provides the data that we will be analysing.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib.pyplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">plt</span>
</pre></div>
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>seed(<span style="color: #666666">999</span>) <span style="color: #408080; font-style: italic"># for reproducibility</span>
a<span style="color: #666666">=0.6</span> <span style="color: #408080; font-style: italic"># biased coin</span>
flips<span style="color: #666666">=</span>np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>rand(<span style="color: #666666">2**12</span>) <span style="color: #408080; font-style: italic"># simulates 4096 coin flips</span>
heads<span style="color: #666666">=</span>flips<span style="color: #666666">&lt;</span>a <span style="color: #408080; font-style: italic"># boolean array, heads[i]=True if flip i is heads</span>
</pre></div>
<p>
In the light of this data, our inference about the fairness of this coin is summarized by the conditional pdf: \( p(H|D,I) \). This is, of course, shorthand for the limiting case of a continuum of propositions for the value of \( H \); that is to say, the probability that \( H \) lies in an infinitesimally narrow range is given by \( p(H|D,I) dH \).
<p>
To estimate this posterior pdf, we need to use Bayes&#8217; theorem <a href="._Bayesian-bs011.html#mjx-eqn-1">(1)</a>. We will ignore the denominator \( p(D|I) \) as it does not involve bias-weighting explicitly, and it will therefore not affect the shape of the desired pdf. At the end we can evaluate the missing constant subsequently from the normalization condition
$$
\begin{equation}
\int_0^1 p(H|D,I) dH = 1.
\tag{2}
\end{equation}
$$
<p>
The prior pdf, \( p(H|I) \), represents what we know about the coin given only the information \( I \) that we are dealing with a &#8216;strange coin&#8217;. We could keep a very open mind about the nature of the coin; a simple probability assignment which reflects this is a uniform, or flat, prior
$$
\begin{equation}
p(H|I) = \left\{ \begin{array}{ll}
1 & 0 \le H \le 1, \\
0 & \mathrm{otherwise}.
\end{array} \right.
\tag{3}
\end{equation}
$$
We will get back later to the choice of prior and its effect on the analysis.
<p>
This prior state of knowledge, or ignorance, is modified by the data through the likelihood function \( p(D|H,I) \). It is a measure of the chance that we would have obtained the data that we actually observed, if the value of the bias-weighting was given (as known). If, in the conditioning information \( I \), we assume that the flips of the coin were independent events, so that the outcome of one did not influence that of another, then the probability of obtaining the data `R heads in N tosses' is given by the binomial distribution (we leave a formal definition of this to a statistics textbook)
$$
\begin{equation}
p(D|H,I) \propto H^R (1-H)^{N-R}.
\tag{4}
\end{equation}
$$
It seems reasonable because \( H \) is the chance of obtaining a head on any flip, and there were \( R \) of them, and \( 1-H \) is the corresponding probability for a tail, of which there were \( N-R \). We note that this binomial distribution also contains a normalization factor, but we will ignore it since it does not depend explicitly on \( H \), the quantity of interest. It will be absorbed by the normalization condition <a href="#mjx-eqn-2">(2)</a>.
<p>
We perform the setup of this Bayesian framework on the computer.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">prior</span>(H):
p<span style="color: #666666">=</span>np<span style="color: #666666">.</span>zeros_like(H)
p[(<span style="color: #666666">0&lt;=</span>x)<span style="color: #666666">&amp;</span>(x<span style="color: #666666">&lt;=1</span>)]<span style="color: #666666">=1</span> <span style="color: #408080; font-style: italic"># allowed range: 0&lt;=H&lt;=1</span>
<span style="color: #008000; font-weight: bold">return</span> p <span style="color: #408080; font-style: italic"># uniform prior</span>
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">likelihood</span>(H,data):
N <span style="color: #666666">=</span> <span style="color: #008000">len</span>(data)
no_of_heads <span style="color: #666666">=</span> <span style="color: #008000">sum</span>(data)
no_of_tails <span style="color: #666666">=</span> N <span style="color: #666666">-</span> no_of_heads
<span style="color: #008000; font-weight: bold">return</span> H<span style="color: #666666">**</span>no_of_heads <span style="color: #666666">*</span> (<span style="color: #666666">1-</span>H)<span style="color: #666666">**</span>no_of_tails
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">posterior</span>(H,data):
p<span style="color: #666666">=</span>prior(H)<span style="color: #666666">*</span>likelihood(H,data)
norm<span style="color: #666666">=</span>np<span style="color: #666666">.</span>trapz(p,H)
<span style="color: #008000; font-weight: bold">return</span> p<span style="color: #666666">/</span>norm
</pre></div>
<p>
The next step is to confront this setup with the simulated data. To get a feel for the result, it is instructive to see how the posterior pdf evolves as we obtain more and more data pertaining to the coin. The results of such an analyses is shown in Fig. <a href="#fig:coinflipping">1</a>.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>x<span style="color: #666666">=</span>np<span style="color: #666666">.</span>linspace(<span style="color: #666666">0</span>,<span style="color: #666666">1</span>,<span style="color: #666666">100</span>)
fig, axs <span style="color: #666666">=</span> plt<span style="color: #666666">.</span>subplots(nrows<span style="color: #666666">=4</span>,ncols<span style="color: #666666">=3</span>,sharex<span style="color: #666666">=</span><span style="color: #008000">True</span>,sharey<span style="color: #666666">=</span><span style="color: #BA2121">&#39;row&#39;</span>)
axs_vec<span style="color: #666666">=</span>np<span style="color: #666666">.</span>reshape(axs,<span style="color: #666666">-1</span>)
axs_vec[<span style="color: #666666">0</span>]<span style="color: #666666">.</span>plot(x,prior(x))
<span style="color: #008000; font-weight: bold">for</span> ndouble <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span>(<span style="color: #666666">11</span>):
ax<span style="color: #666666">=</span>axs_vec[<span style="color: #666666">1+</span>ndouble]
ax<span style="color: #666666">.</span>plot(x,posterior(x,heads[:<span style="color: #666666">2**</span>ndouble]))
ax<span style="color: #666666">.</span>text(<span style="color: #666666">0.1</span>, <span style="color: #666666">0.8</span>, <span style="color: #BA2121">&#39;$N={0}$&#39;</span><span style="color: #666666">.</span>format(<span style="color: #666666">2**</span>ndouble), transform<span style="color: #666666">=</span>ax<span style="color: #666666">.</span>transAxes)
<span style="color: #008000; font-weight: bold">for</span> row <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span>(<span style="color: #666666">4</span>): axs[row,<span style="color: #666666">0</span>]<span style="color: #666666">.</span>set_ylabel(<span style="color: #BA2121">&#39;$p(H|D_\mathrm{obs},I)$&#39;</span>)
<span style="color: #008000; font-weight: bold">for</span> col <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span>(<span style="color: #666666">3</span>): axs[<span style="color: #666666">-1</span>,col]<span style="color: #666666">.</span>set_xlabel(<span style="color: #BA2121">&#39;$H$&#39;</span>)
</pre></div>
<p>
<center> <!-- figure label: --> <div id="fig:coinflipping"></div> <!-- FIGURE -->
<hr class="figure">
<center><p class="caption">Figure 1: The evolution of the posterior pdf for the bias-weighting of a coin, as the number of data available increases. The figure on the top left-hand corner of each panel shows the number of data included in the analysis. <!-- caption label: fig:coinflipping --> </p></center>
<p><img src="fig/coinflipping_fig_1.png" align="bottom" width=500></p>
</center>
<p>
The panel in the top left-hand corner shows the posterior pdf for \( H \) given no data, i.e., it is the same as the prior pdf of Eq. <a href="#mjx-eqn-3">(3)</a>. It indicates that we have no more reason to believe that the coin is fair than we have to think that it is double-headed, double-tailed, or of any other intermediate bias-weighting.
<p>
The first flip is obviously tails. At this point we have no evidence that the coin has a side with heads, as indicated by the pdf going to zero as \( H \to 1 \). The second flip is obviously heads and we have now excluded both extreme options \( H=0 \) (double-tailed) and \( H=1 \) (double-headed). We can note that the posterior at this point has the simple form \( p(H|D,I) = H(1-H) \) for \( 0 \le H \le 1 \).
<p>
The remainder of Fig. <a href="#fig:coinflipping">1</a> shows how the posterior pdf evolves as the number of data analysed becomes larger and larger. We see that the position of the maximum moves around, but that the amount by which it does so decreases with the increasing number of observations. The width of the posterior pdf also becomes narrower with more data, indicating that we are becoming increasingly confident in our estimate of the bias-weighting. For the coin in this example, the best estimate of \( H \) eventually converges to 0.6, which, of course, was the value chosen to simulate the flips.
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Bayesian-bs016.html">&laquo;</a></li>
<li><a href="._Bayesian-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._Bayesian-bs009.html">10</a></li>
<li><a href="._Bayesian-bs010.html">11</a></li>
<li><a href="._Bayesian-bs011.html">12</a></li>
<li><a href="._Bayesian-bs012.html">13</a></li>
<li><a href="._Bayesian-bs013.html">14</a></li>
<li><a href="._Bayesian-bs014.html">15</a></li>
<li><a href="._Bayesian-bs015.html">16</a></li>
<li><a href="._Bayesian-bs016.html">17</a></li>
<li class="active"><a href="._Bayesian-bs017.html">18</a></li>
<li><a href="._Bayesian-bs018.html">19</a></li>
<li><a href="._Bayesian-bs019.html">20</a></li>
<li><a href="._Bayesian-bs020.html">21</a></li>
<li><a href="._Bayesian-bs021.html">22</a></li>
<li><a href="._Bayesian-bs022.html">23</a></li>
<li><a href="._Bayesian-bs018.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/Bayesian/html/._Bayesian-bs018.html
+208
View File
@@ -0,0 +1,208 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks">
<title>Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
('The Bayesian recipe', 2, None, '___sec4'),
("Bayes' theorem", 2, None, '___sec5'),
("The friends of Bayes' theorem", 2, None, '___sec6'),
('Inference With Parametric Models', 2, None, '___sec7'),
('Illustrative examples with python code', 2, None, '___sec8'),
('Example: Is this a fair coin?', 2, None, '___sec9'),
('A few words on different priors', 2, None, '___sec10'),
('Bayesian parameter estimation (single parameter)',
2,
None,
'___sec11'),
('Example: Measured flux from a star', 3, None, '___sec12'),
('Simple Photon Counts: Frequentist Approach',
3,
None,
'___sec13'),
('Simple Photon Counts: Bayesian Approach', 3, None, '___sec14'),
('A note about priors', 3, None, '___sec15'),
('Simple Photon Counts: Bayesian approach in practice',
3,
None,
'___sec16'),
('Best estimates and confidence intervals', 3, None, '___sec17'),
('Simple Photon Counts: Best estimates and confidence intervals',
3,
None,
'___sec18'),
('Bayesian parameter estimation (multiple parameters, '
'covariance)',
2,
None,
'___sec19'),
('Bayesian model selection', 2, None, '___sec20')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Bayesian-bs.html">Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>Why Bayesian Statistics?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs002.html#___sec1" style="font-size: 80%;"><b>Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs003.html#___sec2" style="font-size: 80%;"><b>Statistical Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs004.html#___sec3" style="font-size: 80%;"><b>Some history</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs010.html#___sec4" style="font-size: 80%;"><b>The Bayesian recipe</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs011.html#___sec5" style="font-size: 80%;"><b>Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs013.html#___sec6" style="font-size: 80%;"><b>The friends of Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs014.html#___sec7" style="font-size: 80%;"><b>Inference With Parametric Models</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs016.html#___sec8" style="font-size: 80%;"><b>Illustrative examples with python code</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs017.html#___sec9" style="font-size: 80%;"><b>Example: Is this a fair coin?</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec10" style="font-size: 80%;"><b>A few words on different priors</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs019.html#___sec11" style="font-size: 80%;"><b>Bayesian parameter estimation (single parameter)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec12" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Example: Measured flux from a star</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec13" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Frequentist Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec14" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec15" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;A note about priors</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec16" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian approach in practice</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec17" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec18" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs021.html#___sec19" style="font-size: 80%;"><b>Bayesian parameter estimation (multiple parameters, covariance)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs022.html#___sec20" style="font-size: 80%;"><b>Bayesian model selection</b></a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0018"></a>
<!-- !split -->
<h2 id="___sec10" class="anchor">A few words on different priors </h2>
<ul>
<li> uniform</li>
<li> Gaussian</li>
<li> Jeffrey's prior</li>
</ul>
Repeat the coin flipping experiment with other priors.
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Bayesian-bs017.html">&laquo;</a></li>
<li><a href="._Bayesian-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._Bayesian-bs010.html">11</a></li>
<li><a href="._Bayesian-bs011.html">12</a></li>
<li><a href="._Bayesian-bs012.html">13</a></li>
<li><a href="._Bayesian-bs013.html">14</a></li>
<li><a href="._Bayesian-bs014.html">15</a></li>
<li><a href="._Bayesian-bs015.html">16</a></li>
<li><a href="._Bayesian-bs016.html">17</a></li>
<li><a href="._Bayesian-bs017.html">18</a></li>
<li class="active"><a href="._Bayesian-bs018.html">19</a></li>
<li><a href="._Bayesian-bs019.html">20</a></li>
<li><a href="._Bayesian-bs020.html">21</a></li>
<li><a href="._Bayesian-bs021.html">22</a></li>
<li><a href="._Bayesian-bs022.html">23</a></li>
<li><a href="._Bayesian-bs019.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/Bayesian/html/._Bayesian-bs019.html
+213
View File
@@ -0,0 +1,213 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks">
<title>Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
('The Bayesian recipe', 2, None, '___sec4'),
("Bayes' theorem", 2, None, '___sec5'),
("The friends of Bayes' theorem", 2, None, '___sec6'),
('Inference With Parametric Models', 2, None, '___sec7'),
('Illustrative examples with python code', 2, None, '___sec8'),
('Example: Is this a fair coin?', 2, None, '___sec9'),
('A few words on different priors', 2, None, '___sec10'),
('Bayesian parameter estimation (single parameter)',
2,
None,
'___sec11'),
('Example: Measured flux from a star', 3, None, '___sec12'),
('Simple Photon Counts: Frequentist Approach',
3,
None,
'___sec13'),
('Simple Photon Counts: Bayesian Approach', 3, None, '___sec14'),
('A note about priors', 3, None, '___sec15'),
('Simple Photon Counts: Bayesian approach in practice',
3,
None,
'___sec16'),
('Best estimates and confidence intervals', 3, None, '___sec17'),
('Simple Photon Counts: Best estimates and confidence intervals',
3,
None,
'___sec18'),
('Bayesian parameter estimation (multiple parameters, '
'covariance)',
2,
None,
'___sec19'),
('Bayesian model selection', 2, None, '___sec20')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Bayesian-bs.html">Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>Why Bayesian Statistics?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs002.html#___sec1" style="font-size: 80%;"><b>Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs003.html#___sec2" style="font-size: 80%;"><b>Statistical Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs004.html#___sec3" style="font-size: 80%;"><b>Some history</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs010.html#___sec4" style="font-size: 80%;"><b>The Bayesian recipe</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs011.html#___sec5" style="font-size: 80%;"><b>Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs013.html#___sec6" style="font-size: 80%;"><b>The friends of Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs014.html#___sec7" style="font-size: 80%;"><b>Inference With Parametric Models</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs016.html#___sec8" style="font-size: 80%;"><b>Illustrative examples with python code</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs017.html#___sec9" style="font-size: 80%;"><b>Example: Is this a fair coin?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs018.html#___sec10" style="font-size: 80%;"><b>A few words on different priors</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec11" style="font-size: 80%;"><b>Bayesian parameter estimation (single parameter)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec12" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Example: Measured flux from a star</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec13" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Frequentist Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec14" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec15" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;A note about priors</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec16" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian approach in practice</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec17" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec18" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs021.html#___sec19" style="font-size: 80%;"><b>Bayesian parameter estimation (multiple parameters, covariance)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs022.html#___sec20" style="font-size: 80%;"><b>Bayesian model selection</b></a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0019"></a>
<!-- !split -->
<h2 id="___sec11" class="anchor">Bayesian parameter estimation (single parameter) </h2>
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
We will now consider the very important task of model parameter estimation using statistical inference.
<!-- begin inline comment -->
<font color="red">(<b>CF 1</b>: maybe stress that model parameters are not random variables, and the meaning of parameter estimation is therefore very different between frequentist and bayesian approaches.)</font>
<!-- end inline comment -->
<p>
Throughout this section we will consider a specific example that involves a model with a single parameter: &quot;Measured flux from a star&quot;.
</div>
</div>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Bayesian-bs018.html">&laquo;</a></li>
<li><a href="._Bayesian-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._Bayesian-bs011.html">12</a></li>
<li><a href="._Bayesian-bs012.html">13</a></li>
<li><a href="._Bayesian-bs013.html">14</a></li>
<li><a href="._Bayesian-bs014.html">15</a></li>
<li><a href="._Bayesian-bs015.html">16</a></li>
<li><a href="._Bayesian-bs016.html">17</a></li>
<li><a href="._Bayesian-bs017.html">18</a></li>
<li><a href="._Bayesian-bs018.html">19</a></li>
<li class="active"><a href="._Bayesian-bs019.html">20</a></li>
<li><a href="._Bayesian-bs020.html">21</a></li>
<li><a href="._Bayesian-bs021.html">22</a></li>
<li><a href="._Bayesian-bs022.html">23</a></li>
<li><a href="._Bayesian-bs020.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/Bayesian/html/._Bayesian-bs020.html
+470
View File
@@ -0,0 +1,470 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks">
<title>Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
('The Bayesian recipe', 2, None, '___sec4'),
("Bayes' theorem", 2, None, '___sec5'),
("The friends of Bayes' theorem", 2, None, '___sec6'),
('Inference With Parametric Models', 2, None, '___sec7'),
('Illustrative examples with python code', 2, None, '___sec8'),
('Example: Is this a fair coin?', 2, None, '___sec9'),
('A few words on different priors', 2, None, '___sec10'),
('Bayesian parameter estimation (single parameter)',
2,
None,
'___sec11'),
('Example: Measured flux from a star', 3, None, '___sec12'),
('Simple Photon Counts: Frequentist Approach',
3,
None,
'___sec13'),
('Simple Photon Counts: Bayesian Approach', 3, None, '___sec14'),
('A note about priors', 3, None, '___sec15'),
('Simple Photon Counts: Bayesian approach in practice',
3,
None,
'___sec16'),
('Best estimates and confidence intervals', 3, None, '___sec17'),
('Simple Photon Counts: Best estimates and confidence intervals',
3,
None,
'___sec18'),
('Bayesian parameter estimation (multiple parameters, '
'covariance)',
2,
None,
'___sec19'),
('Bayesian model selection', 2, None, '___sec20')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Bayesian-bs.html">Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>Why Bayesian Statistics?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs002.html#___sec1" style="font-size: 80%;"><b>Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs003.html#___sec2" style="font-size: 80%;"><b>Statistical Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs004.html#___sec3" style="font-size: 80%;"><b>Some history</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs010.html#___sec4" style="font-size: 80%;"><b>The Bayesian recipe</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs011.html#___sec5" style="font-size: 80%;"><b>Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs013.html#___sec6" style="font-size: 80%;"><b>The friends of Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs014.html#___sec7" style="font-size: 80%;"><b>Inference With Parametric Models</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs016.html#___sec8" style="font-size: 80%;"><b>Illustrative examples with python code</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs017.html#___sec9" style="font-size: 80%;"><b>Example: Is this a fair coin?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs018.html#___sec10" style="font-size: 80%;"><b>A few words on different priors</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs019.html#___sec11" style="font-size: 80%;"><b>Bayesian parameter estimation (single parameter)</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec12" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Example: Measured flux from a star</a></li>
<!-- navigation toc: --> <li><a href="#___sec13" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Frequentist Approach</a></li>
<!-- navigation toc: --> <li><a href="#___sec14" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian Approach</a></li>
<!-- navigation toc: --> <li><a href="#___sec15" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;A note about priors</a></li>
<!-- navigation toc: --> <li><a href="#___sec16" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian approach in practice</a></li>
<!-- navigation toc: --> <li><a href="#___sec17" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="#___sec18" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs021.html#___sec19" style="font-size: 80%;"><b>Bayesian parameter estimation (multiple parameters, covariance)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs022.html#___sec20" style="font-size: 80%;"><b>Bayesian model selection</b></a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0020"></a>
<!-- !split -->
<h3 id="___sec12" class="anchor">Example: Measured flux from a star </h3>
Adapted from the blog <a href="http://jakevdp.github.io" target="_self">Pythonic Perambulations</a> by Jake VanderPlas.
<p>
Imagine that we point our telescope to the sky, and observe the light coming from a single star. For the time being, we'll assume that the star's true flux is constant with time, i.e. that is it has a fixed value \( F_\mathrm{true} \) (we'll also ignore effects like sky noise and other sources of systematic error). We'll assume that we perform a series of \( N \) measurements with our telescope, where the ith measurement reports the observed photon flux \( F_i \) and error \( e_i \) <button type="button" class="btn btn-primary btn-xs" rel="tooltip" data-placement="top" title="We'll make the reasonable assumption that errors are Gaussian. In a Frequentist perspective, \( e_i \) is the standard deviation of the results of a single measurement event in the limit of repetitions of that event. In the Bayesian perspective, \( e_i \) is the standard deviation of the (Gaussian) probability distribution describing our knowledge of that particular measurement given its observed value."><a href="#def_footnote_2" id="link_footnote_2" style="color: white">2</a></button>.
The question is, given this set of measurements \( D = \{F_i, e_i\} \), what is our best estimate of the true flux \( F_\mathrm{true} \)?
<p id="def_footnote_2"><a href="#link_footnote_2"><b>2:</b></a> We'll make the reasonable assumption that errors are Gaussian. In a Frequentist perspective, \( e_i \) is the standard deviation of the results of a single measurement event in the limit of repetitions of <em>that event</em>. In the Bayesian perspective, \( e_i \) is the standard deviation of the (Gaussian) probability distribution describing our knowledge of that particular measurement given its observed value.</p>
<p>
Because the measurements are number counts, a Poisson distribution is a good approximation to the measurement process:
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>seed(<span style="color: #666666">1</span>) <span style="color: #408080; font-style: italic"># for repeatability</span>
F_true <span style="color: #666666">=</span> <span style="color: #666666">1000</span> <span style="color: #408080; font-style: italic"># true flux, say number of photons measured in 1 second</span>
N <span style="color: #666666">=</span> <span style="color: #666666">50</span> <span style="color: #408080; font-style: italic"># number of measurements</span>
F <span style="color: #666666">=</span> stats<span style="color: #666666">.</span>poisson(F_true)<span style="color: #666666">.</span>rvs(N)
<span style="color: #408080; font-style: italic"># N measurements of the flux</span>
e <span style="color: #666666">=</span> np<span style="color: #666666">.</span>sqrt(F) <span style="color: #408080; font-style: italic"># errors on Poisson counts estimated via square root</span>
</pre></div>
<p>
Now let's make a simple visualization of the &quot;observed&quot; data, see Fig. <a href="#fig:flux">2</a>.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>fig, ax <span style="color: #666666">=</span> plt<span style="color: #666666">.</span>subplots()
ax<span style="color: #666666">.</span>errorbar(F, np<span style="color: #666666">.</span>arange(N), xerr<span style="color: #666666">=</span>e, fmt<span style="color: #666666">=</span><span style="color: #BA2121">&#39;ok&#39;</span>, ecolor<span style="color: #666666">=</span><span style="color: #BA2121">&#39;gray&#39;</span>, alpha<span style="color: #666666">=0.5</span>)
ax<span style="color: #666666">.</span>vlines([F_true], <span style="color: #666666">0</span>, N, linewidth<span style="color: #666666">=5</span>, alpha<span style="color: #666666">=0.2</span>)
ax<span style="color: #666666">.</span>set_xlabel(<span style="color: #BA2121">&quot;Flux&quot;</span>);ax<span style="color: #666666">.</span>set_ylabel(<span style="color: #BA2121">&quot;measurement number&quot;</span>);
</pre></div>
<p>
<center> <!-- figure label: --> <div id="fig:flux"></div> <!-- FIGURE -->
<hr class="figure">
<center><p class="caption">Figure 2: Single photon counts (flux measurements). <!-- caption label: fig:flux --> </p></center>
<p><img src="fig/singlephotoncount_fig_1.png" align="bottom" width=400></p>
</center>
<p>
These measurements each have a different error \( e_i \) which is estimated from Poisson statistics using the standard square-root rule. In this toy example we already know the true flux \( F_\mathrm{true} \), but the question is this: given our measurements and errors, what is our best estimate of the true flux?
<p>
Let's take a look at the frequentist and Bayesian approaches to solving this.
<h3 id="___sec13" class="anchor">Simple Photon Counts: Frequentist Approach </h3>
We'll start with the classical frequentist maximum likelihood approach. Given a single observation \( D_i = (F_i, e_i) \), we can compute the probability distribution of the measurement given the true flux Ftrue given our assumption of Gaussian errors
$$
\begin{equation}
p(D_i | F_\mathrm{true}, I) = \frac{1}{\sqrt{2\pi e_i^2}} \exp \left( \frac{-(F_i-F_\mathrm{true})^2}{2e_i^2} \right).
\tag{5}
\end{equation}
$$
This should be read &quot;the probability of \( D_i \) given \( F_\mathrm{true} \)
equals ...&quot;. You should recognize this as a normal distribution with mean \( F_\mathrm{true} \) and standard deviation \( e_i \).
<p>
We construct the <em>likelihood function</em> by computing the product of the probabilities for each data point
$$
\begin{equation}
\mathcal{L}(D | F_\mathrm{true}, I) = \prod_{i=1}^N p(D_i | F_\mathrm{true}, I),
\tag{6}
\end{equation}
$$
here \( D = \{D_i\} \) represents the entire set of measurements. Because the value of the likelihood can become very small, it is often more convenient to instead compute the log-likelihood. Combining the previous two equations and computing the log, we have
$$
\begin{equation}
\log\mathcal{L} = -\frac{1}{2} \sum_{i=1}^N \left[ \log(2\pi e_i^2) + \frac{(F_i-F_\mathrm{true})^2}{e_i^2} \right].
\tag{7}
\end{equation}
$$
<p>
What we'd like to do is determine \( F_\mathrm{true} \) such that the likelihood is maximized. For this simple problem, the maximization can be computed analytically (i.e. by setting \( d\log\mathcal{L}/d F_\mathrm{true} = 0 \)). This results in the following observed estimate of \( F_\mathrm{true} \)
$$
\begin{equation}
F_\mathrm{est} = \sum_{i=1}^N w_i F_i; \quad w_i = 1/e_i^2.
\tag{8}
\end{equation}
$$
Notice that in the special case of all errors \( e_i \) being equal, this reduces to
$$
\begin{equation}
F_\mathrm{est} = \frac{1}{N} \sum_{i=1} F_i.
\tag{9}
\end{equation}
$$
That is, in agreement with intuition, \( F_\mathrm{est} \) is simply the mean of the observed data when errors are equal.
<p>
We can go further and ask what the error of our estimate is. In the frequentist approach, this can be accomplished by fitting a Gaussian approximation to the likelihood curve at maximum; in this simple case this can also be solved analytically (the sum of Gaussians is also a Gaussian). It can be shown that the standard deviation of this Gaussian approximation is
$$
\begin{equation}
\sigma_\mathrm{est} = \sum_{i=1}^N w_i.
\tag{10}
\end{equation}
$$
These results are fairly simple calculations; let's evaluate them for our toy dataset:
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>w<span style="color: #666666">=1./</span>e<span style="color: #666666">**2</span>
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;&quot;&quot;</span>
<span style="color: #BA2121">F_true = {0}</span>
<span style="color: #BA2121">F_est = {1:.0f} +/- {2:.0f} (based on {3} measurements) &quot;&quot;&quot;</span>\
<span style="color: #666666">.</span>format(F_true, (w <span style="color: #666666">*</span> F)<span style="color: #666666">.</span>sum() <span style="color: #666666">/</span> w<span style="color: #666666">.</span>sum(), w<span style="color: #666666">.</span>sum() <span style="color: #666666">**</span> <span style="color: #666666">-0.5</span>, N))
</pre></div>
<p>
<code>F_true = 1000</code> <br />
<code>F_est = 998 +/- 4 (based on 50 measurements)</code> <br />
<p>
We find that for 50 measurements of the flux, our estimate has an error of about 0.4% and is consistent with the input value.
<h3 id="___sec14" class="anchor">Simple Photon Counts: Bayesian Approach </h3>
The Bayesian approach, as you might expect, begins and ends with probabilities. Our hypothesis is that the star has a constant flux \( F_\mathrm{true} \). It recognizes that what we fundamentally want to compute is our knowledge of the parameters in question given the data and other information (such as our knowledge of uncertainties for the observed values), i.e. in this case, \( p(F_\mathrm{true} | D,I) \).
Note that this formulation of the problem is fundamentally contrary to the frequentist philosophy, which says that probabilities have no meaning for model parameters like \( F_\mathrm{true} \). Nevertheless, within the Bayesian philosophy this is perfectly acceptable.
<p>
To compute this result, Bayesians next apply Bayes' Theorem <a href="._Bayesian-bs011.html#mjx-eqn-1">(1)</a>.
If we set the prior \( p(F_\mathrm{true}|I) \propto 1 \) (a flat prior), we find
\( p(F_\mathrm{true}|D,I) \propto p(D | F_\mathrm{true},I) \equiv \mathcal{L}(D | F_\mathrm{true},I) \)
and the Bayesian probability is maximized at precisely the same value as the frequentist result! So despite the philosophical differences, we see that (for this simple problem at least) the Bayesian and frequentist point estimates are equivalent.
<h3 id="___sec15" class="anchor">A note about priors </h3>
The prior allows inclusion of other information into the computation, which becomes very useful in cases where multiple measurement strategies are being combined to constrain a single model. The necessity to specify a prior, however, is one of the more controversial pieces of Bayesian analysis.
A frequentist will point out that the prior is problematic when no true prior information is available. Though it might seem straightforward to use a noninformative prior like the flat prior mentioned above, there are some <a href="http://normaldeviate.wordpress.com/2013/07/13/lost-causes-in-statistics-ii-noninformative- priors/comment-page-1/" target="_self">surprisingly subtleties</a> involved. It turns out that in many situations, a truly noninformative prior does not exist! Frequentists point out that the subjective choice of a prior which necessarily biases your result has no place in statistical data analysis.
A Bayesian would counter that frequentism doesn't solve this problem, but simply skirts the question. Frequentism can often be viewed as simply a special case of the Bayesian approach for some (implicit) choice of the prior: a Bayesian would say that it's better to make this implicit choice explicit, even if the choice might include some subjectivity.
<h3 id="___sec16" class="anchor">Simple Photon Counts: Bayesian approach in practice </h3>
Leaving these philosophical debates aside for the time being, let's address how Bayesian results are generally computed in practice. For a one parameter problem like the one considered here, it's as simple as computing the posterior probability \( p(F_\mathrm{true} | D,I) \) as a function of \( F_\mathrm{true} \): this is the distribution reflecting our knowledge of the parameter \( F_\mathrm{true} \).
But as the dimension of the model grows, this direct approach becomes increasingly intractable. For this reason, Bayesian calculations often depend on sampling methods such as Markov Chain Monte Carlo (MCMC). For this practical example, let us apply an MCMC approach using Dan Foreman-Mackey's <a href="http://dan.iel.fm/emcee/current/" target="_self">emcee</a> package. Keep in mind here that the goal is to generate a set of points drawn from the posterior probability distribution, and to use those points to determine the answer we seek.
To perform this MCMC, we start by defining Python functions for the prior \( p(F_\mathrm{true} | I) \), the likelihood \( p(D | F_\mathrm{true},I) \), and the posterior \( p(F_\mathrm{true} | D,I) \), noting that none of these need be properly normalized. Our model here is one-dimensional, but to handle multi-dimensional models we'll define the model in terms of an array of parameters \( \boldsymbol{\alpha} \), which in this case is \( \boldsymbol{\alpha} = [F_\mathrm{true}] \)
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">log_prior</span>(alpha):
<span style="color: #008000; font-weight: bold">return</span> <span style="color: #666666">0</span> <span style="color: #408080; font-style: italic"># flat prior</span>
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">log_likelihood</span>(alpha, F, e):
<span style="color: #008000; font-weight: bold">return</span> <span style="color: #666666">-0.5</span> <span style="color: #666666">*</span> np<span style="color: #666666">.</span>sum(np<span style="color: #666666">.</span>log(<span style="color: #666666">2</span> <span style="color: #666666">*</span> np<span style="color: #666666">.</span>pi <span style="color: #666666">*</span> e <span style="color: #666666">**</span> <span style="color: #666666">2</span>) \
<span style="color: #666666">+</span> (F <span style="color: #666666">-</span> alpha[<span style="color: #666666">0</span>]) <span style="color: #666666">**</span> <span style="color: #666666">2</span> <span style="color: #666666">/</span> e <span style="color: #666666">**</span> <span style="color: #666666">2</span>)
<span style="color: #008000; font-weight: bold">def</span> <span style="color: #0000FF">log_posterior</span>(alpha, F, e):
<span style="color: #008000; font-weight: bold">return</span> log_prior(alpha) <span style="color: #666666">+</span> log_likelihood(alpha, F, e)
</pre></div>
<p>
Now we set up the problem, including generating some random starting guesses for the multiple chains of points.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>ndim <span style="color: #666666">=</span> <span style="color: #666666">1</span> <span style="color: #408080; font-style: italic"># number of parameters in the model</span>
nwalkers <span style="color: #666666">=</span> <span style="color: #666666">50</span> <span style="color: #408080; font-style: italic"># number of MCMC walkers</span>
nburn <span style="color: #666666">=</span> <span style="color: #666666">1000</span> <span style="color: #408080; font-style: italic"># &quot;burn-in&quot; period to let chains stabilize</span>
nsteps <span style="color: #666666">=</span> <span style="color: #666666">2000</span> <span style="color: #408080; font-style: italic"># number of MCMC steps to take</span>
<span style="color: #408080; font-style: italic"># we&#39;ll start at random locations between 0 and 2000</span>
starting_guesses <span style="color: #666666">=</span> <span style="color: #666666">2000</span> <span style="color: #666666">*</span> np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>rand(nwalkers, ndim)
sampler <span style="color: #666666">=</span> emcee<span style="color: #666666">.</span>EnsembleSampler(nwalkers, ndim, log_posterior, args<span style="color: #666666">=</span>[F,e])
sampler<span style="color: #666666">.</span>run_mcmc(starting_guesses, nsteps)
<span style="color: #408080; font-style: italic"># Shape of sampler.chain = (nwalkers, nsteps, ndim)</span>
<span style="color: #408080; font-style: italic"># Flatten the sampler chain and discard burn-in points:</span>
samples <span style="color: #666666">=</span> sampler<span style="color: #666666">.</span>chain[:, nburn:, :]<span style="color: #666666">.</span>reshape((<span style="color: #666666">-1</span>, ndim))
</pre></div>
<p>
If this all worked correctly, the array sample should contain a series of 50,000 points drawn from the posterior. Let's plot them and check. See results in Fig. <a href="#fig:flux-bayesian">3</a>.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>fig, ax <span style="color: #666666">=</span> plt<span style="color: #666666">.</span>subplots()
ax<span style="color: #666666">.</span>hist(samples, bins<span style="color: #666666">=50</span>, histtype<span style="color: #666666">=</span><span style="color: #BA2121">&quot;stepfilled&quot;</span>, alpha<span style="color: #666666">=0.3</span>, normed<span style="color: #666666">=</span><span style="color: #008000">True</span>)
ax<span style="color: #666666">.</span>set_xlabel(<span style="color: #BA2121">r&#39;$F_\mathrm{est}$&#39;</span>)
ax<span style="color: #666666">.</span>set_ylabel(<span style="color: #BA2121">r&#39;$p(F_\mathrm{est}|D,I)$&#39;</span>)
</pre></div>
<p>
<center> <!-- figure label: --> <div id="fig:flux-bayesian"></div> <!-- FIGURE -->
<hr class="figure">
<center><p class="caption">Figure 3: Bayesian posterior pdf (represented by a histogram of MCMC samples) from flux measurements. <!-- caption label: fig:flux-bayesian --> </p></center>
<p><img src="fig/singlephotoncount_fig_2.png" align="bottom" width=400></p>
</center>
<h3 id="___sec17" class="anchor">Best estimates and confidence intervals </h3>
The posterior distribution from our Bayesian data analysis is the key quantity that encodes our inference about the values of the model parameters, given the data and the relevant background information. Often, however, we wish to summarize this result with just a few numbers: the best estimate and a measure of its reliability.
<p>
There are a few different options for this. The choice of the most appropriate one depends mainly on the shape of the posterior distribution:
<p>
<em>Symmetric posterior pdfs</em>: Since the probability (density) associated with any particular value of the parameter is a measure of how much we believe that it lies in the neighbourhood of that point, our best estimate is given by the maximum of the posterior pdf. If we denote the quantity of interest by \( X \), with a posterior pdf \( P =p(X|D,I) \), then the best estimate of its value \( X_0 \) is given by the condition \( dP/dX|_{X=X_0}=0 \). Strictly speaking, we should also check the sign of the second derivative to ensure that \( X_0 \) represents a maximum.
<p>
To obtain a measure of the reliability of this best estimate, we need to look at the width or spread of the posterior pdf about \( X_0 \). When considering the behaviour of any function in the neighbourhood of a particular point, it is often helpful to carry out a Taylor series expansion; this is simply a standard tool for (locally) approximating a complicated function by a low-order polynomial. The linear term is zero at the maximum and the quadratic term is often the dominating one determining the width of the posterior pdf. Ignoring all the higher-order terms we arrive at the Gaussian approximation
$$
\begin{equation}
p(X|D,I) \approx \frac{1}{\sigma\sqrt{2\pi}} \exp \left[ -\frac{(x-\mu)^2}{2\sigma^2} \right],
\tag{11}
\end{equation}
$$
where the mean \( \mu = X_0 \) and the variance \( \sigma = \left( - \left. \frac{d^2L}{dX^2} \right|_{X_0} \right)^{-1/2} \), where \( L \) is the logarithm of the posterior \( P \). Our inference about the quantity of interest is conveyed very concisely, therefore, by the statement \( X = X_0 \pm \sigma \), and
$$
$$
p(X_0-\sigma < X < X_0+\sigma | D,I) = \int_{X_0-\sigma}^{X_0+\sigma} p(X|D,I) dX \approx 0.67.
$$
$$
<p>
<em>Asymmetric posterior pdfs</em>: While the maximum of the posterior (\( X_0 \)) can still be regarded as giving the best estimate, the true value is now more likely to be on one side of this rather than the other. Alternatively one can compute the mean value, \( \langle X \rangle = \int X p(X|D,I) dX \), although this tends to overemphasise very long tails. The best option is probably a compromise that can be employed when having access to a large sample from the posterior (as provided by an MCMC), namely to give the median of this ensamble.
<p>
Furthermore, the concept of an error-bar does not seem appropriate in this case, as it implicitly entails the idea of symmetry. A good way of expressing the reliability with which a parameter can be inferred, for an asymmetric posterior pdf, is rather through a <em>confidence interval</em>. Since the area under the posterior pdf between \( X_1 \) and \( X_2 \) is proportional to how much we believe that \( X \) lies in that range, the shortest interval that encloses 67% of the area represents a sensible measure of the uncertainty of the estimate. Obviously we can choose to provide some other degree-of-belief that we think is relevant for the case at hand. Assuming that the posterior pdf has been normalized, to have unit area, we need to find \( X_1 \) and \( X_2 \) such that:
$$
$$
p(X_1 < X < X_2 | D,I) = \int_{X_1}^{X_2} p(X|D,I) dX \approx 0.67,
$$
$$
where the difference \( X_2 - X_1 \) is as small as possible. The region \( X_1 < X < X_2 \) is then called the shortest 67% confidence interval.
<p>
<em>Multimodal posterior pdfs</em>: We can sometimes obtain posteriors which are multimodal; i.e. contains several disconnected regions with large probabilities. There is no difficulty when one of the maxima is very much larger than the others: we can simply ignore the subsidiary solutions, to a good approximation, and concentrate on the global maximum. The problem arises when there are several maxima of comparable magnitude. What do we now mean by a best estimate, and how should we quantify its reliability? The idea of a best estimate and an error-bar, or even a confidence interval, is merely an attempt to summarize the posterior with just two or three numbers; sometimes this just can&#8217;t be done, and so these concepts are not valid. For the bimodal case we might be able to characterize the posterior in terms of a few numbers: two best estimates and their associated error-bars, or disjoint confidence intervals. For a general multimodal pdf, the most honest thing we can do is just display the posterior itself.
<h3 id="___sec18" class="anchor">Simple Photon Counts: Best estimates and confidence intervals </h3>
To compute these numbers for our example, you would run:
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>sampper<span style="color: #666666">=</span>np<span style="color: #666666">.</span>percentile(samples, [<span style="color: #666666">2.5</span>, <span style="color: #666666">16.5</span>, <span style="color: #666666">50</span>, <span style="color: #666666">83.5</span>, <span style="color: #666666">97.5</span>],axis<span style="color: #666666">=0</span>)<span style="color: #666666">.</span>flatten()
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;&quot;&quot;</span>
<span style="color: #BA2121">F_true = {0}</span>
<span style="color: #BA2121">Based on {1} measurements the posterior point estimates are:</span>
<span style="color: #BA2121">...F_est = {2:.0f} +/- {3:.0f}</span>
<span style="color: #BA2121">or using credible intervals:</span>
<span style="color: #BA2121">...F_est = {4:.0f} (posterior median) </span>
<span style="color: #BA2121">...F_est in [{5:.0f}, {6:.0f}] (67</span><span style="color: #BB6688; font-weight: bold">% c</span><span style="color: #BA2121">redible interval) </span>
<span style="color: #BA2121">...F_est in [{7:.0f}, {8:.0f}] (95</span><span style="color: #BB6688; font-weight: bold">% c</span><span style="color: #BA2121">redible interval) &quot;&quot;&quot;</span>\
<span style="color: #666666">.</span>format(F_true, N, np<span style="color: #666666">.</span>mean(samples), np<span style="color: #666666">.</span>std(samples), \
sampper[<span style="color: #666666">2</span>], sampper[<span style="color: #666666">1</span>], sampper[<span style="color: #666666">3</span>], sampper[<span style="color: #666666">0</span>], sampper[<span style="color: #666666">4</span>]))
</pre></div>
<p>
<code>F_true = 1000</code> <br />
<code>Based on 50 measurements the posterior point estimates are:</code> <br />
<code>...F_est = 998 +/- 4</code> <br />
<code>or using credible intervals:</code> <br />
<code>...F_est = 998 (posterior median)</code> <br />
<code>...F_est in [993, 1002] (67% credible interval)</code> <br />
<code>...F_est in [989, 1006] (95% credible interval)</code> <br />
<p>
In this particular example, the posterior pdf is actually a Gaussian (since it is constructed as a product of Gaussians), and the mean and variance from the quadratic approximation will agree exactly with the frequentist approach.
<p>
From this final result you might come away with the impression that the Bayesian method is unnecessarily complicated, and in this case it certainly is. Using an MCMC sampler to characterize a one-dimensional normal distribution is a bit like using the Death Star to destroy a beach ball, but we did this here because it demonstrates an approach that can scale to complicated posteriors in many, many dimensions, and can provide nice results in more complicated situations where an analytic likelihood approach is not possible.
<p>
Furthermore, as data and models grow in complexity, the two approaches can diverge greatly.
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Bayesian-bs019.html">&laquo;</a></li>
<li><a href="._Bayesian-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._Bayesian-bs012.html">13</a></li>
<li><a href="._Bayesian-bs013.html">14</a></li>
<li><a href="._Bayesian-bs014.html">15</a></li>
<li><a href="._Bayesian-bs015.html">16</a></li>
<li><a href="._Bayesian-bs016.html">17</a></li>
<li><a href="._Bayesian-bs017.html">18</a></li>
<li><a href="._Bayesian-bs018.html">19</a></li>
<li><a href="._Bayesian-bs019.html">20</a></li>
<li class="active"><a href="._Bayesian-bs020.html">21</a></li>
<li><a href="._Bayesian-bs021.html">22</a></li>
<li><a href="._Bayesian-bs022.html">23</a></li>
<li><a href="._Bayesian-bs021.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/Bayesian/html/._Bayesian-bs021.html
+210
View File
@@ -0,0 +1,210 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks">
<title>Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
('The Bayesian recipe', 2, None, '___sec4'),
("Bayes' theorem", 2, None, '___sec5'),
("The friends of Bayes' theorem", 2, None, '___sec6'),
('Inference With Parametric Models', 2, None, '___sec7'),
('Illustrative examples with python code', 2, None, '___sec8'),
('Example: Is this a fair coin?', 2, None, '___sec9'),
('A few words on different priors', 2, None, '___sec10'),
('Bayesian parameter estimation (single parameter)',
2,
None,
'___sec11'),
('Example: Measured flux from a star', 3, None, '___sec12'),
('Simple Photon Counts: Frequentist Approach',
3,
None,
'___sec13'),
('Simple Photon Counts: Bayesian Approach', 3, None, '___sec14'),
('A note about priors', 3, None, '___sec15'),
('Simple Photon Counts: Bayesian approach in practice',
3,
None,
'___sec16'),
('Best estimates and confidence intervals', 3, None, '___sec17'),
('Simple Photon Counts: Best estimates and confidence intervals',
3,
None,
'___sec18'),
('Bayesian parameter estimation (multiple parameters, '
'covariance)',
2,
None,
'___sec19'),
('Bayesian model selection', 2, None, '___sec20')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Bayesian-bs.html">Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>Why Bayesian Statistics?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs002.html#___sec1" style="font-size: 80%;"><b>Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs003.html#___sec2" style="font-size: 80%;"><b>Statistical Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs004.html#___sec3" style="font-size: 80%;"><b>Some history</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs010.html#___sec4" style="font-size: 80%;"><b>The Bayesian recipe</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs011.html#___sec5" style="font-size: 80%;"><b>Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs013.html#___sec6" style="font-size: 80%;"><b>The friends of Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs014.html#___sec7" style="font-size: 80%;"><b>Inference With Parametric Models</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs016.html#___sec8" style="font-size: 80%;"><b>Illustrative examples with python code</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs017.html#___sec9" style="font-size: 80%;"><b>Example: Is this a fair coin?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs018.html#___sec10" style="font-size: 80%;"><b>A few words on different priors</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs019.html#___sec11" style="font-size: 80%;"><b>Bayesian parameter estimation (single parameter)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec12" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Example: Measured flux from a star</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec13" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Frequentist Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec14" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec15" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;A note about priors</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec16" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian approach in practice</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec17" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec18" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="#___sec19" style="font-size: 80%;"><b>Bayesian parameter estimation (multiple parameters, covariance)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs022.html#___sec20" style="font-size: 80%;"><b>Bayesian model selection</b></a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0021"></a>
<!-- !split -->
<h2 id="___sec19" class="anchor">Bayesian parameter estimation (multiple parameters, covariance) </h2>
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
<ul>
<li> multidimensional posterior pdf:s</li>
<li> nuisance parameters (e.g. background subtraction?)</li>
<li> corner plots, covariance, correlations</li>
<li> best example?</li>
</ul>
</div>
</div>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Bayesian-bs020.html">&laquo;</a></li>
<li><a href="._Bayesian-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._Bayesian-bs013.html">14</a></li>
<li><a href="._Bayesian-bs014.html">15</a></li>
<li><a href="._Bayesian-bs015.html">16</a></li>
<li><a href="._Bayesian-bs016.html">17</a></li>
<li><a href="._Bayesian-bs017.html">18</a></li>
<li><a href="._Bayesian-bs018.html">19</a></li>
<li><a href="._Bayesian-bs019.html">20</a></li>
<li><a href="._Bayesian-bs020.html">21</a></li>
<li class="active"><a href="._Bayesian-bs021.html">22</a></li>
<li><a href="._Bayesian-bs022.html">23</a></li>
<li><a href="._Bayesian-bs022.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/Bayesian/html/._Bayesian-bs022.html
+206
View File
@@ -0,0 +1,206 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks">
<title>Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
('The Bayesian recipe', 2, None, '___sec4'),
("Bayes' theorem", 2, None, '___sec5'),
("The friends of Bayes' theorem", 2, None, '___sec6'),
('Inference With Parametric Models', 2, None, '___sec7'),
('Illustrative examples with python code', 2, None, '___sec8'),
('Example: Is this a fair coin?', 2, None, '___sec9'),
('A few words on different priors', 2, None, '___sec10'),
('Bayesian parameter estimation (single parameter)',
2,
None,
'___sec11'),
('Example: Measured flux from a star', 3, None, '___sec12'),
('Simple Photon Counts: Frequentist Approach',
3,
None,
'___sec13'),
('Simple Photon Counts: Bayesian Approach', 3, None, '___sec14'),
('A note about priors', 3, None, '___sec15'),
('Simple Photon Counts: Bayesian approach in practice',
3,
None,
'___sec16'),
('Best estimates and confidence intervals', 3, None, '___sec17'),
('Simple Photon Counts: Best estimates and confidence intervals',
3,
None,
'___sec18'),
('Bayesian parameter estimation (multiple parameters, '
'covariance)',
2,
None,
'___sec19'),
('Bayesian model selection', 2, None, '___sec20')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="Bayesian-bs.html">Data Analysis and Machine Learning: Elements of Bayesian theory and Bayesian Neural Networks</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>Why Bayesian Statistics?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs002.html#___sec1" style="font-size: 80%;"><b>Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs003.html#___sec2" style="font-size: 80%;"><b>Statistical Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs004.html#___sec3" style="font-size: 80%;"><b>Some history</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs010.html#___sec4" style="font-size: 80%;"><b>The Bayesian recipe</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs011.html#___sec5" style="font-size: 80%;"><b>Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs013.html#___sec6" style="font-size: 80%;"><b>The friends of Bayes' theorem</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs014.html#___sec7" style="font-size: 80%;"><b>Inference With Parametric Models</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs016.html#___sec8" style="font-size: 80%;"><b>Illustrative examples with python code</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs017.html#___sec9" style="font-size: 80%;"><b>Example: Is this a fair coin?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs018.html#___sec10" style="font-size: 80%;"><b>A few words on different priors</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs019.html#___sec11" style="font-size: 80%;"><b>Bayesian parameter estimation (single parameter)</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec12" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Example: Measured flux from a star</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec13" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Frequentist Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec14" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian Approach</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec15" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;A note about priors</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec16" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Bayesian approach in practice</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec17" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs020.html#___sec18" style="font-size: 80%;">&nbsp;&nbsp;&nbsp;Simple Photon Counts: Best estimates and confidence intervals</a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs021.html#___sec19" style="font-size: 80%;"><b>Bayesian parameter estimation (multiple parameters, covariance)</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec20" style="font-size: 80%;"><b>Bayesian model selection</b></a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0022"></a>
<!-- !split -->
<h2 id="___sec20" class="anchor">Bayesian model selection </h2>
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
<ul>
<li> Bayesian evidence</li>
<li> Occam's razor</li>
<li> Best example? How many spectral lines are there?</li>
</ul>
</div>
</div>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._Bayesian-bs021.html">&laquo;</a></li>
<li><a href="._Bayesian-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._Bayesian-bs014.html">15</a></li>
<li><a href="._Bayesian-bs015.html">16</a></li>
<li><a href="._Bayesian-bs016.html">17</a></li>
<li><a href="._Bayesian-bs017.html">18</a></li>
<li><a href="._Bayesian-bs018.html">19</a></li>
<li><a href="._Bayesian-bs019.html">20</a></li>
<li><a href="._Bayesian-bs020.html">21</a></li>
<li><a href="._Bayesian-bs021.html">22</a></li>
<li class="active"><a href="._Bayesian-bs022.html">23</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/Bayesian/html/reveal.js/css/theme/template/mixins.scss
+29
View File
@@ -0,0 +1,29 @@
@mixin vertical-gradient( $top, $bottom ) {
background: $top;
background: -moz-linear-gradient( top, $top 0%, $bottom 100% );
background: -webkit-gradient( linear, left top, left bottom, color-stop(0%,$top), color-stop(100%,$bottom) );
background: -webkit-linear-gradient( top, $top 0%, $bottom 100% );
background: -o-linear-gradient( top, $top 0%, $bottom 100% );
background: -ms-linear-gradient( top, $top 0%, $bottom 100% );
background: linear-gradient( top, $top 0%, $bottom 100% );
}
@mixin horizontal-gradient( $top, $bottom ) {
background: $top;
background: -moz-linear-gradient( left, $top 0%, $bottom 100% );
background: -webkit-gradient( linear, left top, right top, color-stop(0%,$top), color-stop(100%,$bottom) );
background: -webkit-linear-gradient( left, $top 0%, $bottom 100% );
background: -o-linear-gradient( left, $top 0%, $bottom 100% );
background: -ms-linear-gradient( left, $top 0%, $bottom 100% );
background: linear-gradient( left, $top 0%, $bottom 100% );
}
@mixin radial-gradient( $outer, $inner, $type: circle ) {
background: $outer;
background: -moz-radial-gradient( center, $type cover, $inner 0%, $outer 100% );
background: -webkit-gradient( radial, center center, 0px, center center, 100%, color-stop(0%,$inner), color-stop(100%,$outer) );
background: -webkit-radial-gradient( center, $type cover, $inner 0%, $outer 100% );
background: -o-radial-gradient( center, $type cover, $inner 0%, $outer 100% );
background: -ms-radial-gradient( center, $type cover, $inner 0%, $outer 100% );
background: radial-gradient( center, $type cover, $inner 0%, $outer 100% );
}
doc/pub/Bayesian/html/reveal.js/css/theme/template/settings.scss
+34
View File
@@ -0,0 +1,34 @@
// Base settings for all themes that can optionally be
// overridden by the super-theme
// Background of the presentation
$backgroundColor: #2b2b2b;
// Primary/body text
$mainFont: 'Lato', sans-serif;
$mainFontSize: 30px; /* changed (by hpl) from 36px; */
$mainColor: #eee;
// Headings
$headingMargin: 0 0 20px 0;
$headingFont: 'League Gothic', Impact, sans-serif;
$headingColor: #eee;
$headingLineHeight: 0.9em;
$headingLetterSpacing: 0.02em;
$headingTextTransform: none; /* changed (by hpl) from uppercase; */
$headingTextShadow: 0px 0px 6px rgba(0,0,0,0.2);
$heading1TextShadow: $headingTextShadow;
// Links and actions
$linkColor: #13DAEC;
$linkColorHover: lighten( $linkColor, 20% );
// Text selection
$selectionBackgroundColor: #FF5E99;
$selectionColor: #fff;
// Generates the presentation background, can be overridden
// to return a background image or gradient
@mixin bodyBackground() {
background: $backgroundColor;
}
doc/pub/Bayesian/html/reveal.js/css/theme/template/theme.scss
+171
View File
@@ -0,0 +1,171 @@
// Base theme template for reveal.js
/*********************************************
* GLOBAL STYLES
*********************************************/
body {
@include bodyBackground();
background-color: $backgroundColor;
}
.reveal {
font-family: $mainFont;
font-size: $mainFontSize;
font-weight: normal;
letter-spacing: -0.02em;
color: $mainColor;
}
::selection {
color: $selectionColor;
background: $selectionBackgroundColor;
text-shadow: none;
}
/*********************************************
* HEADERS
*********************************************/
.reveal h1,
.reveal h2,
.reveal h3,
.reveal h4,
.reveal h5,
.reveal h6 {
margin: $headingMargin;
color: $headingColor;
font-family: $headingFont;
line-height: $headingLineHeight;
letter-spacing: $headingLetterSpacing;
text-transform: $headingTextTransform;
text-shadow: $headingTextShadow;
}
.reveal h1 {
line-height: 1.2em; /* added by hpl */
text-shadow: $heading1TextShadow;
}
/*********************************************
* LINKS
*********************************************/
.reveal a:not(.image) {
color: $linkColor;
text-decoration: none;
-webkit-transition: color .15s ease;
-moz-transition: color .15s ease;
-ms-transition: color .15s ease;
-o-transition: color .15s ease;
transition: color .15s ease;
}
.reveal a:not(.image):hover {
color: $linkColorHover;
text-shadow: none;
border: none;
}
.reveal .roll span:after {
color: #fff;
background: darken( $linkColor, 15% );
}
/*********************************************
* IMAGES
*********************************************/
.reveal section img {
margin: 15px 0px;
background: rgba(255,255,255,0.12);
border: 4px solid $mainColor;
box-shadow: 0 0 10px rgba(0, 0, 0, 0.15);
-webkit-transition: all .2s linear;
-moz-transition: all .2s linear;
-ms-transition: all .2s linear;
-o-transition: all .2s linear;
transition: all .2s linear;
}
.reveal a:hover img {
background: rgba(255,255,255,0.2);
border-color: $linkColor;
box-shadow: 0 0 20px rgba(0, 0, 0, 0.55);
}
/*********************************************
* NAVIGATION CONTROLS
*********************************************/
.reveal .controls div.navigate-left,
.reveal .controls div.navigate-left.enabled {
border-right-color: $linkColor;
}
.reveal .controls div.navigate-right,
.reveal .controls div.navigate-right.enabled {
border-left-color: $linkColor;
}
.reveal .controls div.navigate-up,
.reveal .controls div.navigate-up.enabled {
border-bottom-color: $linkColor;
}
.reveal .controls div.navigate-down,
.reveal .controls div.navigate-down.enabled {
border-top-color: $linkColor;
}
.reveal .controls div.navigate-left.enabled:hover {
border-right-color: $linkColorHover;
}
.reveal .controls div.navigate-right.enabled:hover {
border-left-color: $linkColorHover;
}
.reveal .controls div.navigate-up.enabled:hover {
border-bottom-color: $linkColorHover;
}
.reveal .controls div.navigate-down.enabled:hover {
border-top-color: $linkColorHover;
}
/*********************************************
* PROGRESS BAR
*********************************************/
.reveal .progress {
background: rgba(0,0,0,0.2);
}
.reveal .progress span {
background: $linkColor;
-webkit-transition: width 800ms cubic-bezier(0.260, 0.860, 0.440, 0.985);
-moz-transition: width 800ms cubic-bezier(0.260, 0.860, 0.440, 0.985);
-ms-transition: width 800ms cubic-bezier(0.260, 0.860, 0.440, 0.985);
-o-transition: width 800ms cubic-bezier(0.260, 0.860, 0.440, 0.985);
transition: width 800ms cubic-bezier(0.260, 0.860, 0.440, 0.985);
}
/*********************************************
* SLIDE NUMBER
*********************************************/
.reveal .slide-number {
color: $linkColor;
}
doc/pub/How2ReadData/html/How2ReadData-bs.html
+1024 -828
View File
File diff suppressed because it is too large Load Diff
doc/pub/How2ReadData/html/How2ReadData-reveal.html
+1002 -828
View File
File diff suppressed because it is too large Load Diff
doc/pub/How2ReadData/html/How2ReadData-solarized.html
+992 -807
View File
File diff suppressed because it is too large Load Diff
doc/pub/How2ReadData/html/How2ReadData.html
+992 -807
View File
File diff suppressed because it is too large Load Diff
doc/pub/How2ReadData/html/fig/pandas.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 101 KiB

doc/pub/How2ReadData/html/reveal.js/css/theme/template/mixins.scss
+29
View File
@@ -0,0 +1,29 @@
@mixin vertical-gradient( $top, $bottom ) {
background: $top;
background: -moz-linear-gradient( top, $top 0%, $bottom 100% );
background: -webkit-gradient( linear, left top, left bottom, color-stop(0%,$top), color-stop(100%,$bottom) );
background: -webkit-linear-gradient( top, $top 0%, $bottom 100% );
background: -o-linear-gradient( top, $top 0%, $bottom 100% );
background: -ms-linear-gradient( top, $top 0%, $bottom 100% );
background: linear-gradient( top, $top 0%, $bottom 100% );
}
@mixin horizontal-gradient( $top, $bottom ) {
background: $top;
background: -moz-linear-gradient( left, $top 0%, $bottom 100% );
background: -webkit-gradient( linear, left top, right top, color-stop(0%,$top), color-stop(100%,$bottom) );
background: -webkit-linear-gradient( left, $top 0%, $bottom 100% );
background: -o-linear-gradient( left, $top 0%, $bottom 100% );
background: -ms-linear-gradient( left, $top 0%, $bottom 100% );
background: linear-gradient( left, $top 0%, $bottom 100% );
}
@mixin radial-gradient( $outer, $inner, $type: circle ) {
background: $outer;
background: -moz-radial-gradient( center, $type cover, $inner 0%, $outer 100% );
background: -webkit-gradient( radial, center center, 0px, center center, 100%, color-stop(0%,$inner), color-stop(100%,$outer) );
background: -webkit-radial-gradient( center, $type cover, $inner 0%, $outer 100% );
background: -o-radial-gradient( center, $type cover, $inner 0%, $outer 100% );
background: -ms-radial-gradient( center, $type cover, $inner 0%, $outer 100% );
background: radial-gradient( center, $type cover, $inner 0%, $outer 100% );
}
doc/pub/How2ReadData/html/reveal.js/css/theme/template/settings.scss
+34
View File
@@ -0,0 +1,34 @@
// Base settings for all themes that can optionally be
// overridden by the super-theme
// Background of the presentation
$backgroundColor: #2b2b2b;
// Primary/body text
$mainFont: 'Lato', sans-serif;
$mainFontSize: 30px; /* changed (by hpl) from 36px; */
$mainColor: #eee;
// Headings
$headingMargin: 0 0 20px 0;
$headingFont: 'League Gothic', Impact, sans-serif;
$headingColor: #eee;
$headingLineHeight: 0.9em;
$headingLetterSpacing: 0.02em;
$headingTextTransform: none; /* changed (by hpl) from uppercase; */
$headingTextShadow: 0px 0px 6px rgba(0,0,0,0.2);
$heading1TextShadow: $headingTextShadow;
// Links and actions
$linkColor: #13DAEC;
$linkColorHover: lighten( $linkColor, 20% );
// Text selection
$selectionBackgroundColor: #FF5E99;
$selectionColor: #fff;
// Generates the presentation background, can be overridden
// to return a background image or gradient
@mixin bodyBackground() {
background: $backgroundColor;
}
doc/pub/How2ReadData/html/reveal.js/css/theme/template/theme.scss
+171
View File
@@ -0,0 +1,171 @@
// Base theme template for reveal.js
/*********************************************
* GLOBAL STYLES
*********************************************/
body {
@include bodyBackground();
background-color: $backgroundColor;
}
.reveal {
font-family: $mainFont;
font-size: $mainFontSize;
font-weight: normal;
letter-spacing: -0.02em;
color: $mainColor;
}
::selection {
color: $selectionColor;
background: $selectionBackgroundColor;
text-shadow: none;
}
/*********************************************
* HEADERS
*********************************************/
.reveal h1,
.reveal h2,
.reveal h3,
.reveal h4,
.reveal h5,
.reveal h6 {
margin: $headingMargin;
color: $headingColor;
font-family: $headingFont;
line-height: $headingLineHeight;
letter-spacing: $headingLetterSpacing;
text-transform: $headingTextTransform;
text-shadow: $headingTextShadow;
}
.reveal h1 {
line-height: 1.2em; /* added by hpl */
text-shadow: $heading1TextShadow;
}
/*********************************************
* LINKS
*********************************************/
.reveal a:not(.image) {
color: $linkColor;
text-decoration: none;
-webkit-transition: color .15s ease;
-moz-transition: color .15s ease;
-ms-transition: color .15s ease;
-o-transition: color .15s ease;
transition: color .15s ease;
}
.reveal a:not(.image):hover {
color: $linkColorHover;
text-shadow: none;
border: none;
}
.reveal .roll span:after {
color: #fff;
background: darken( $linkColor, 15% );
}
/*********************************************
* IMAGES
*********************************************/
.reveal section img {
margin: 15px 0px;
background: rgba(255,255,255,0.12);
border: 4px solid $mainColor;
box-shadow: 0 0 10px rgba(0, 0, 0, 0.15);
-webkit-transition: all .2s linear;
-moz-transition: all .2s linear;
-ms-transition: all .2s linear;
-o-transition: all .2s linear;
transition: all .2s linear;
}
.reveal a:hover img {
background: rgba(255,255,255,0.2);
border-color: $linkColor;
box-shadow: 0 0 20px rgba(0, 0, 0, 0.55);
}
/*********************************************
* NAVIGATION CONTROLS
*********************************************/
.reveal .controls div.navigate-left,
.reveal .controls div.navigate-left.enabled {
border-right-color: $linkColor;
}
.reveal .controls div.navigate-right,
.reveal .controls div.navigate-right.enabled {
border-left-color: $linkColor;
}
.reveal .controls div.navigate-up,
.reveal .controls div.navigate-up.enabled {
border-bottom-color: $linkColor;
}
.reveal .controls div.navigate-down,
.reveal .controls div.navigate-down.enabled {
border-top-color: $linkColor;
}
.reveal .controls div.navigate-left.enabled:hover {
border-right-color: $linkColorHover;
}
.reveal .controls div.navigate-right.enabled:hover {
border-left-color: $linkColorHover;
}
.reveal .controls div.navigate-up.enabled:hover {
border-bottom-color: $linkColorHover;
}
.reveal .controls div.navigate-down.enabled:hover {
border-top-color: $linkColorHover;
}
/*********************************************
* PROGRESS BAR
*********************************************/
.reveal .progress {
background: rgba(0,0,0,0.2);
}
.reveal .progress span {
background: $linkColor;
-webkit-transition: width 800ms cubic-bezier(0.260, 0.860, 0.440, 0.985);
-moz-transition: width 800ms cubic-bezier(0.260, 0.860, 0.440, 0.985);
-ms-transition: width 800ms cubic-bezier(0.260, 0.860, 0.440, 0.985);
-o-transition: width 800ms cubic-bezier(0.260, 0.860, 0.440, 0.985);
transition: width 800ms cubic-bezier(0.260, 0.860, 0.440, 0.985);
}
/*********************************************
* SLIDE NUMBER
*********************************************/
.reveal .slide-number {
color: $linkColor;
}
doc/pub/How2ReadData/html/reveal.js/demo.html
+410
View File
@@ -0,0 +1,410 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>reveal.js – The HTML Presentation Framework</title>
<meta name="description" content="A framework for easily creating beautiful presentations using HTML">
<meta name="author" content="Hakim El Hattab">
<meta name="apple-mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent">
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no">
<link rel="stylesheet" href="css/reveal.css">
<link rel="stylesheet" href="css/theme/black.css" id="theme">
<!-- Theme used for syntax highlighting of code -->
<link rel="stylesheet" href="lib/css/zenburn.css">
<!-- Printing and PDF exports -->
<script>
var link = document.createElement( 'link' );
link.rel = 'stylesheet';
link.type = 'text/css';
link.href = window.location.search.match( /print-pdf/gi ) ? 'css/print/pdf.css' : 'css/print/paper.css';
document.getElementsByTagName( 'head' )[0].appendChild( link );
</script>
<!--[if lt IE 9]>
<script src="lib/js/html5shiv.js"></script>
<![endif]-->
</head>
<body>
<div class="reveal">
<!-- Any section element inside of this container is displayed as a slide -->
<div class="slides">
<section>
<h1>Reveal.js</h1>
<h3>The HTML Presentation Framework</h3>
<p>
<small>Created by <a href="http://hakim.se">Hakim El Hattab</a> and <a href="https://github.com/hakimel/reveal.js/graphs/contributors">contributors</a></small>
</p>
</section>
<section>
<h2>Hello There</h2>
<p>
reveal.js enables you to create beautiful interactive slide decks using HTML. This presentation will show you examples of what it can do.
</p>
</section>
<!-- Example of nested vertical slides -->
<section>
<section>
<h2>Vertical Slides</h2>
<p>Slides can be nested inside of each other.</p>
<p>Use the <em>Space</em> key to navigate through all slides.</p>
<br>
<a href="#" class="navigate-down">
<img width="178" height="238" data-src="https://s3.amazonaws.com/hakim-static/reveal-js/arrow.png" alt="Down arrow">
</a>
</section>
<section>
<h2>Basement Level 1</h2>
<p>Nested slides are useful for adding additional detail underneath a high level horizontal slide.</p>
</section>
<section>
<h2>Basement Level 2</h2>
<p>That's it, time to go back up.</p>
<br>
<a href="#/2">
<img width="178" height="238" data-src="https://s3.amazonaws.com/hakim-static/reveal-js/arrow.png" alt="Up arrow" style="transform: rotate(180deg); -webkit-transform: rotate(180deg);">
</a>
</section>
</section>
<section>
<h2>Slides</h2>
<p>
Not a coder? Not a problem. There's a fully-featured visual editor for authoring these, try it out at <a href="https://slides.com" target="_blank">https://slides.com</a>.
</p>
</section>
<section>
<h2>Point of View</h2>
<p>
Press <strong>ESC</strong> to enter the slide overview.
</p>
<p>
Hold down alt and click on any element to zoom in on it using <a href="http://lab.hakim.se/zoom-js">zoom.js</a>. Alt + click anywhere to zoom back out.
</p>
</section>
<section>
<h2>Touch Optimized</h2>
<p>
Presentations look great on touch devices, like mobile phones and tablets. Simply swipe through your slides.
</p>
</section>
<section data-markdown>
<script type="text/template">
## Markdown support
Write content using inline or external Markdown.
Instructions and more info available in the [readme](https://github.com/hakimel/reveal.js#markdown).
```
<section data-markdown>
## Markdown support
Write content using inline or external Markdown.
Instructions and more info available in the [readme](https://github.com/hakimel/reveal.js#markdown).
</section>
```
</script>
</section>
<section>
<section id="fragments">
<h2>Fragments</h2>
<p>Hit the next arrow...</p>
<p class="fragment">... to step through ...</p>
<p><span class="fragment">... a</span> <span class="fragment">fragmented</span> <span class="fragment">slide.</span></p>
<aside class="notes">
This slide has fragments which are also stepped through in the notes window.
</aside>
</section>
<section>
<h2>Fragment Styles</h2>
<p>There's different types of fragments, like:</p>
<p class="fragment grow">grow</p>
<p class="fragment shrink">shrink</p>
<p class="fragment fade-out">fade-out</p>
<p class="fragment fade-up">fade-up (also down, left and right!)</p>
<p class="fragment current-visible">current-visible</p>
<p>Highlight <span class="fragment highlight-red">red</span> <span class="fragment highlight-blue">blue</span> <span class="fragment highlight-green">green</span></p>
</section>
</section>
<section id="transitions">
<h2>Transition Styles</h2>
<p>
You can select from different transitions, like: <br>
<a href="?transition=none#/transitions">None</a> -
<a href="?transition=fade#/transitions">Fade</a> -
<a href="?transition=slide#/transitions">Slide</a> -
<a href="?transition=convex#/transitions">Convex</a> -
<a href="?transition=concave#/transitions">Concave</a> -
<a href="?transition=zoom#/transitions">Zoom</a>
</p>
</section>
<section id="themes">
<h2>Themes</h2>
<p>
reveal.js comes with a few themes built in: <br>
<!-- Hacks to swap themes after the page has loaded. Not flexible and only intended for the reveal.js demo deck. -->
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/black.css'); return false;">Black (default)</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/white.css'); return false;">White</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/league.css'); return false;">League</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/sky.css'); return false;">Sky</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/beige.css'); return false;">Beige</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/simple.css'); return false;">Simple</a> <br>
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/serif.css'); return false;">Serif</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/blood.css'); return false;">Blood</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/night.css'); return false;">Night</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/moon.css'); return false;">Moon</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/solarized.css'); return false;">Solarized</a>
</p>
</section>
<section>
<section data-background="#dddddd">
<h2>Slide Backgrounds</h2>
<p>
Set <code>data-background="#dddddd"</code> on a slide to change the background color. All CSS color formats are supported.
</p>
<a href="#" class="navigate-down">
<img width="178" height="238" data-src="https://s3.amazonaws.com/hakim-static/reveal-js/arrow.png" alt="Down arrow">
</a>
</section>
<section data-background="https://s3.amazonaws.com/hakim-static/reveal-js/image-placeholder.png">
<h2>Image Backgrounds</h2>
<pre><code class="hljs">&lt;section data-background="image.png"&gt;</code></pre>
</section>
<section data-background="https://s3.amazonaws.com/hakim-static/reveal-js/image-placeholder.png" data-background-repeat="repeat" data-background-size="100px">
<h2>Tiled Backgrounds</h2>
<pre><code class="hljs" style="word-wrap: break-word;">&lt;section data-background="image.png" data-background-repeat="repeat" data-background-size="100px"&gt;</code></pre>
</section>
<section data-background-video="https://s3.amazonaws.com/static.slid.es/site/homepage/v1/homepage-video-editor.mp4,https://s3.amazonaws.com/static.slid.es/site/homepage/v1/homepage-video-editor.webm" data-background-color="#000000">
<div style="background-color: rgba(0, 0, 0, 0.9); color: #fff; padding: 20px;">
<h2>Video Backgrounds</h2>
<pre><code class="hljs" style="word-wrap: break-word;">&lt;section data-background-video="video.mp4,video.webm"&gt;</code></pre>
</div>
</section>
<section data-background="http://i.giphy.com/90F8aUepslB84.gif">
<h2>... and GIFs!</h2>
</section>
</section>
<section data-transition="slide" data-background="#4d7e65" data-background-transition="zoom">
<h2>Background Transitions</h2>
<p>
Different background transitions are available via the backgroundTransition option. This one's called "zoom".
</p>
<pre><code class="hljs">Reveal.configure({ backgroundTransition: 'zoom' })</code></pre>
</section>
<section data-transition="slide" data-background="#b5533c" data-background-transition="zoom">
<h2>Background Transitions</h2>
<p>
You can override background transitions per-slide.
</p>
<pre><code class="hljs" style="word-wrap: break-word;">&lt;section data-background-transition="zoom"&gt;</code></pre>
</section>
<section>
<h2>Pretty Code</h2>
<pre><code class="hljs" data-trim contenteditable>
function linkify( selector ) {
if( supports3DTransforms ) {
var nodes = document.querySelectorAll( selector );
for( var i = 0, len = nodes.length; i &lt; len; i++ ) {
var node = nodes[i];
if( !node.className ) {
node.className += ' roll';
}
}
}
}
</code></pre>
<p>Code syntax highlighting courtesy of <a href="http://softwaremaniacs.org/soft/highlight/en/description/">highlight.js</a>.</p>
</section>
<section>
<h2>Marvelous List</h2>
<ul>
<li>No order here</li>
<li>Or here</li>
<li>Or here</li>
<li>Or here</li>
</ul>
</section>
<section>
<h2>Fantastic Ordered List</h2>
<ol>
<li>One is smaller than...</li>
<li>Two is smaller than...</li>
<li>Three!</li>
</ol>
</section>
<section>
<h2>Tabular Tables</h2>
<table>
<thead>
<tr>
<th>Item</th>
<th>Value</th>
<th>Quantity</th>
</tr>
</thead>
<tbody>
<tr>
<td>Apples</td>
<td>$1</td>
<td>7</td>
</tr>
<tr>
<td>Lemonade</td>
<td>$2</td>
<td>18</td>
</tr>
<tr>
<td>Bread</td>
<td>$3</td>
<td>2</td>
</tr>
</tbody>
</table>
</section>
<section>
<h2>Clever Quotes</h2>
<p>
These guys come in two forms, inline: <q cite="http://searchservervirtualization.techtarget.com/definition/Our-Favorite-Technology-Quotations">The nice thing about standards is that there are so many to choose from</q> and block:
</p>
<blockquote cite="http://searchservervirtualization.techtarget.com/definition/Our-Favorite-Technology-Quotations">
&ldquo;For years there has been a theory that millions of monkeys typing at random on millions of typewriters would
reproduce the entire works of Shakespeare. The Internet has proven this theory to be untrue.&rdquo;
</blockquote>
</section>
<section>
<h2>Intergalactic Interconnections</h2>
<p>
You can link between slides internally,
<a href="#/2/3">like this</a>.
</p>
</section>
<section>
<h2>Speaker View</h2>
<p>There's a <a href="https://github.com/hakimel/reveal.js#speaker-notes">speaker view</a>. It includes a timer, preview of the upcoming slide as well as your speaker notes.</p>
<p>Press the <em>S</em> key to try it out.</p>
<aside class="notes">
Oh hey, these are some notes. They'll be hidden in your presentation, but you can see them if you open the speaker notes window (hit 's' on your keyboard).
</aside>
</section>
<section>
<h2>Export to PDF</h2>
<p>Presentations can be <a href="https://github.com/hakimel/reveal.js#pdf-export">exported to PDF</a>, here's an example:</p>
<iframe data-src="https://www.slideshare.net/slideshow/embed_code/42840540" width="445" height="355" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" style="border:3px solid #666; margin-bottom:5px; max-width: 100%;" allowfullscreen> </iframe>
</section>
<section>
<h2>Global State</h2>
<p>
Set <code>data-state="something"</code> on a slide and <code>"something"</code>
will be added as a class to the document element when the slide is open. This lets you
apply broader style changes, like switching the page background.
</p>
</section>
<section data-state="customevent">
<h2>State Events</h2>
<p>
Additionally custom events can be triggered on a per slide basis by binding to the <code>data-state</code> name.
</p>
<pre><code class="javascript" data-trim contenteditable style="font-size: 18px;">
Reveal.addEventListener( 'customevent', function() {
console.log( '"customevent" has fired' );
} );
</code></pre>
</section>
<section>
<h2>Take a Moment</h2>
<p>
Press B or . on your keyboard to pause the presentation. This is helpful when you're on stage and want to take distracting slides off the screen.
</p>
</section>
<section>
<h2>Much more</h2>
<ul>
<li>Right-to-left support</li>
<li><a href="https://github.com/hakimel/reveal.js#api">Extensive JavaScript API</a></li>
<li><a href="https://github.com/hakimel/reveal.js#auto-sliding">Auto-progression</a></li>
<li><a href="https://github.com/hakimel/reveal.js#parallax-background">Parallax backgrounds</a></li>
<li><a href="https://github.com/hakimel/reveal.js#keyboard-bindings">Custom keyboard bindings</a></li>
</ul>
</section>
<section style="text-align: left;">
<h1>THE END</h1>
<p>
- <a href="https://slides.com">Try the online editor</a> <br>
- <a href="https://github.com/hakimel/reveal.js">Source code &amp; documentation</a>
</p>
</section>
</div>
</div>
<script src="lib/js/head.min.js"></script>
<script src="js/reveal.js"></script>
<script>
// More info https://github.com/hakimel/reveal.js#configuration
Reveal.initialize({
controls: true,
progress: true,
history: true,
center: true,
transition: 'slide', // none/fade/slide/convex/concave/zoom
// More info https://github.com/hakimel/reveal.js#dependencies
dependencies: [
{ src: 'lib/js/classList.js', condition: function() { return !document.body.classList; } },
{ src: 'plugin/markdown/marked.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } },
{ src: 'plugin/markdown/markdown.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } },
{ src: 'plugin/highlight/highlight.js', async: true, callback: function() { hljs.initHighlightingOnLoad(); } },
{ src: 'plugin/search/search.js', async: true },
{ src: 'plugin/zoom-js/zoom.js', async: true },
{ src: 'plugin/notes/notes.js', async: true }
]
});
</script>
</body>
</html>
doc/pub/How2ReadData/html/reveal.js/plugin/multiplex/package.json
+19
View File
@@ -0,0 +1,19 @@
{
"name": "reveal-js-multiplex",
"version": "1.0.0",
"description": "reveal.js multiplex server",
"homepage": "http://revealjs.com",
"scripts": {
"start": "node index.js"
},
"engines": {
"node": "~4.1.1"
},
"dependencies": {
"express": "~4.13.3",
"grunt-cli": "~0.1.13",
"mustache": "~2.2.1",
"socket.io": "~1.3.7"
},
"license": "MIT"
}
doc/pub/How2ReadData/html/reveal.js/test/simple.md
+12
View File
@@ -0,0 +1,12 @@
## Slide 1.1
```js
var a = 1;
```
## Slide 1.2
## Slide 2
doc/pub/How2ReadData/html/reveal.js/test/test-markdown-external.html
+36
View File
@@ -0,0 +1,36 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>reveal.js - Test Markdown</title>
<link rel="stylesheet" href="../css/reveal.css">
<link rel="stylesheet" href="qunit-1.12.0.css">
</head>
<body style="overflow: auto;">
<div id="qunit"></div>
<div id="qunit-fixture"></div>
<div class="reveal" style="display: none;">
<div class="slides">
<section data-markdown="simple.md" data-separator="^\n\n\n" data-separator-vertical="^\n\n"></section>
</div>
</div>
<script src="../lib/js/head.min.js"></script>
<script src="../js/reveal.js"></script>
<script src="../plugin/highlight/highlight.js"></script>
<script src="../plugin/markdown/marked.js"></script>
<script src="../plugin/markdown/markdown.js"></script>
<script src="qunit-1.12.0.js"></script>
<script src="test-markdown-external.js"></script>
</body>
</html>
doc/pub/How2ReadData/html/reveal.js/test/test-markdown-external.js
+24
View File
@@ -0,0 +1,24 @@
Reveal.addEventListener( 'ready', function() {
QUnit.module( 'Markdown' );
test( 'Vertical separator', function() {
strictEqual( document.querySelectorAll( '.reveal .slides>section>section' ).length, 2, 'found two slides' );
});
test( 'Horizontal separator', function() {
strictEqual( document.querySelectorAll( '.reveal .slides>section' ).length, 2, 'found two slides' );
});
test( 'Language highlighter', function() {
strictEqual( document.querySelectorAll( '.hljs-keyword' ).length, 1, 'got rendered highlight tag.' );
strictEqual( document.querySelector( '.hljs-keyword' ).innerHTML, 'var', 'the same keyword: var.' );
});
} );
Reveal.initialize();
doc/pub/How2ReadData/html/reveal.js/test/test-markdown-options.html
+41
View File
@@ -0,0 +1,41 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>reveal.js - Test Markdown Options</title>
<link rel="stylesheet" href="../css/reveal.css">
<link rel="stylesheet" href="qunit-1.12.0.css">
</head>
<body style="overflow: auto;">
<div id="qunit"></div>
<div id="qunit-fixture"></div>
<div class="reveal" style="display: none;">
<div class="slides">
<section data-markdown>
<script type="text/template">
## Testing Markdown Options
This "slide" should contain 'smart' quotes.
</script>
</section>
</div>
</div>
<script src="../lib/js/head.min.js"></script>
<script src="../js/reveal.js"></script>
<script src="qunit-1.12.0.js"></script>
<script src="test-markdown-options.js"></script>
</body>
</html>
doc/pub/How2ReadData/html/reveal.js/test/test-markdown-options.js
+26
View File
@@ -0,0 +1,26 @@
Reveal.addEventListener( 'ready', function() {
QUnit.module( 'Markdown' );
test( 'Options are set', function() {
strictEqual( marked.defaults.smartypants, true );
});
test( 'Smart quotes are activated', function() {
var text = document.querySelector( '.reveal .slides>section>p' ).textContent;
strictEqual( /['"]/.test( text ), false );
strictEqual( /[“”‘’]/.test( text ), true );
});
} );
Reveal.initialize({
dependencies: [
{ src: '../plugin/markdown/marked.js' },
{ src: '../plugin/markdown/markdown.js' },
],
markdown: {
smartypants: true
}
});
doc/pub/How2ReadData/html/src/Hudson_Bay.py~
+43
View File
@@ -0,0 +1,43 @@
import numpy as np
import matplotlib.pyplot as plt
def solver(m, H0, L0, dt, a, b, c, d, t0):
"""Solve the difference equations for H and L over m years
with time step dt (measured in years."""
num_intervals = int(m/float(dt))
t = np.linspace(t0, t0 + m, num_intervals+1)
H = np.zeros(t.size)
L = np.zeros(t.size)
print 'Init:', H0, L0, dt
H[0] = H0
L[0] = L0
for n in range(0, len(t)-1):
H[n+1] = H[n] + a*dt*H[n] - b*dt*H[n]*L[n]
L[n+1] = L[n] + d*dt*H[n]*L[n] - c*dt*L[n]
return H, L, t
# Load in data file
data = np.loadtxt('Hudson_Bay.csv', delimiter=',', skiprows=1)
# Make arrays containing x-axis and hares and lynx populations
t_e = data[:,0]
H_e = data[:,1]
L_e = data[:,2]
# Simulate using the model
H, L, t = solver(m=20, H0=34.91, L0=3.857, dt=0.1,
a=0.4807, b=0.02482, c=0.9272, d=0.02756,
t0=1900)
# Visualize simulations and data
plt.plot(t_e, H_e, 'b-+', t_e, L_e, 'r-o', t, H, 'm--', t, L, 'k--')
plt.xlabel('Year')
plt.ylabel('Numbers of hares and lynx')
plt.axis([1900, 1920, 0, 140])
plt.title(r'Population of hares and lynx 1900-1920 (x1000)')
plt.legend(('H_e', 'L_e', 'H', 'L'), loc='upper left')
plt.savefig('Hudson_Bay_sim.pdf')
plt.savefig('Hudson_Bay_sim.png')
plt.show()
doc/pub/How2ReadData/html/src/plot_Hudson.py~
+19
View File
@@ -0,0 +1,19 @@
import numpy as np
from matplotlib import pyplot as plt
# Load in data file
data = np.loadtxt('src/Hudson_Bay.dat', delimiter=',', skiprows=1)
# Make arrays containing x-axis and hares and lynx populations
year = data[:,0]
hares = data[:,1]
lynx = data[:,2]
plt.plot(year, hares ,'b-+', year, lynx, 'r-o')
plt.axis([1900,1920,0, 100.0])
plt.xlabel(r'Year')
plt.ylabel(r'Numbers of hares and lynx ')
plt.legend(('Hares','Lynx'), loc='upper right')
plt.title(r'Population of hares and lynx from 1900-1920 (x1000)}')
plt.savefig('Hudson_Bay_data.pdf')
plt.savefig('Hudson_Bay_data.png')
plt.show()
doc/pub/How2ReadData/ipynb/.DS_Store
Vendored
BIN
View File
Binary file not shown.
doc/pub/How2ReadData/ipynb/How2ReadData.ipynb
+1439 -1086
View File
File diff suppressed because it is too large Load Diff
doc/pub/How2ReadData/ipynb/fig/pandas.jpg
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 101 KiB

doc/pub/How2ReadData/ipynb/ipynb-How2ReadData-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/How2ReadData/ipynb/src/Hudson_Bay.py~
+43
View File
@@ -0,0 +1,43 @@
import numpy as np
import matplotlib.pyplot as plt
def solver(m, H0, L0, dt, a, b, c, d, t0):
"""Solve the difference equations for H and L over m years
with time step dt (measured in years."""
num_intervals = int(m/float(dt))
t = np.linspace(t0, t0 + m, num_intervals+1)
H = np.zeros(t.size)
L = np.zeros(t.size)
print 'Init:', H0, L0, dt
H[0] = H0
L[0] = L0
for n in range(0, len(t)-1):
H[n+1] = H[n] + a*dt*H[n] - b*dt*H[n]*L[n]
L[n+1] = L[n] + d*dt*H[n]*L[n] - c*dt*L[n]
return H, L, t
# Load in data file
data = np.loadtxt('Hudson_Bay.csv', delimiter=',', skiprows=1)
# Make arrays containing x-axis and hares and lynx populations
t_e = data[:,0]
H_e = data[:,1]
L_e = data[:,2]
# Simulate using the model
H, L, t = solver(m=20, H0=34.91, L0=3.857, dt=0.1,
a=0.4807, b=0.02482, c=0.9272, d=0.02756,
t0=1900)
# Visualize simulations and data
plt.plot(t_e, H_e, 'b-+', t_e, L_e, 'r-o', t, H, 'm--', t, L, 'k--')
plt.xlabel('Year')
plt.ylabel('Numbers of hares and lynx')
plt.axis([1900, 1920, 0, 140])
plt.title(r'Population of hares and lynx 1900-1920 (x1000)')
plt.legend(('H_e', 'L_e', 'H', 'L'), loc='upper left')
plt.savefig('Hudson_Bay_sim.pdf')
plt.savefig('Hudson_Bay_sim.png')
plt.show()
doc/pub/How2ReadData/ipynb/src/plot_Hudson.py~
+19
View File
@@ -0,0 +1,19 @@
import numpy as np
from matplotlib import pyplot as plt
# Load in data file
data = np.loadtxt('src/Hudson_Bay.dat', delimiter=',', skiprows=1)
# Make arrays containing x-axis and hares and lynx populations
year = data[:,0]
hares = data[:,1]
lynx = data[:,2]
plt.plot(year, hares ,'b-+', year, lynx, 'r-o')
plt.axis([1900,1920,0, 100.0])
plt.xlabel(r'Year')
plt.ylabel(r'Numbers of hares and lynx ')
plt.legend(('Hares','Lynx'), loc='upper right')
plt.title(r'Population of hares and lynx from 1900-1920 (x1000)}')
plt.savefig('Hudson_Bay_data.pdf')
plt.savefig('Hudson_Bay_data.png')
plt.show()
doc/pub/How2ReadData/pdf/How2ReadData-minted.pdf
BIN
View File
Binary file not shown.
doc/pub/Intro2Course/html/reveal.js/plugin/leap/leap.js
+159
View File
File diff suppressed because one or more lines are too long
doc/pub/Intro2Course/html/reveal.js/plugin/remotes/remotes.js
+39
View File
@@ -0,0 +1,39 @@
/**
* Touch-based remote controller for your presentation courtesy
* of the folks at http://remotes.io
*/
(function(window){
/**
* Detects if we are dealing with a touch enabled device (with some false positives)
* Borrowed from modernizr: https://github.com/Modernizr/Modernizr/blob/master/feature-detects/touch.js
*/
var hasTouch = (function(){
return ('ontouchstart' in window) || window.DocumentTouch && document instanceof DocumentTouch;
})();
/**
* Detects if notes are enable and the current page is opened inside an /iframe
* this prevents loading Remotes.io several times
*/
var isNotesAndIframe = (function(){
return window.RevealNotes && !(self == top);
})();
if(!hasTouch && !isNotesAndIframe){
head.ready( 'remotes.ne.min.js', function() {
new Remotes("preview")
.on("swipe-left", function(e){ Reveal.right(); })
.on("swipe-right", function(e){ Reveal.left(); })
.on("swipe-up", function(e){ Reveal.down(); })
.on("swipe-down", function(e){ Reveal.up(); })
.on("tap", function(e){ Reveal.next(); })
.on("zoom-out", function(e){ Reveal.toggleOverview(true); })
.on("zoom-in", function(e){ Reveal.toggleOverview(false); })
;
} );
head.js('https://hakim-static.s3.amazonaws.com/reveal-js/remotes.ne.min.js');
}
})(window);
doc/pub/Linalg/html/reveal.js/css/theme/template/mixins.scss
+29
View File
@@ -0,0 +1,29 @@
@mixin vertical-gradient( $top, $bottom ) {
background: $top;
background: -moz-linear-gradient( top, $top 0%, $bottom 100% );
background: -webkit-gradient( linear, left top, left bottom, color-stop(0%,$top), color-stop(100%,$bottom) );
background: -webkit-linear-gradient( top, $top 0%, $bottom 100% );
background: -o-linear-gradient( top, $top 0%, $bottom 100% );
background: -ms-linear-gradient( top, $top 0%, $bottom 100% );
background: linear-gradient( top, $top 0%, $bottom 100% );
}
@mixin horizontal-gradient( $top, $bottom ) {
background: $top;
background: -moz-linear-gradient( left, $top 0%, $bottom 100% );
background: -webkit-gradient( linear, left top, right top, color-stop(0%,$top), color-stop(100%,$bottom) );
background: -webkit-linear-gradient( left, $top 0%, $bottom 100% );
background: -o-linear-gradient( left, $top 0%, $bottom 100% );
background: -ms-linear-gradient( left, $top 0%, $bottom 100% );
background: linear-gradient( left, $top 0%, $bottom 100% );
}
@mixin radial-gradient( $outer, $inner, $type: circle ) {
background: $outer;
background: -moz-radial-gradient( center, $type cover, $inner 0%, $outer 100% );
background: -webkit-gradient( radial, center center, 0px, center center, 100%, color-stop(0%,$inner), color-stop(100%,$outer) );
background: -webkit-radial-gradient( center, $type cover, $inner 0%, $outer 100% );
background: -o-radial-gradient( center, $type cover, $inner 0%, $outer 100% );
background: -ms-radial-gradient( center, $type cover, $inner 0%, $outer 100% );
background: radial-gradient( center, $type cover, $inner 0%, $outer 100% );
}
doc/pub/Linalg/html/reveal.js/css/theme/template/settings.scss
+34
View File
@@ -0,0 +1,34 @@
// Base settings for all themes that can optionally be
// overridden by the super-theme
// Background of the presentation
$backgroundColor: #2b2b2b;
// Primary/body text
$mainFont: 'Lato', sans-serif;
$mainFontSize: 30px; /* changed (by hpl) from 36px; */
$mainColor: #eee;
// Headings
$headingMargin: 0 0 20px 0;
$headingFont: 'League Gothic', Impact, sans-serif;
$headingColor: #eee;
$headingLineHeight: 0.9em;
$headingLetterSpacing: 0.02em;
$headingTextTransform: none; /* changed (by hpl) from uppercase; */
$headingTextShadow: 0px 0px 6px rgba(0,0,0,0.2);
$heading1TextShadow: $headingTextShadow;
// Links and actions
$linkColor: #13DAEC;
$linkColorHover: lighten( $linkColor, 20% );
// Text selection
$selectionBackgroundColor: #FF5E99;
$selectionColor: #fff;
// Generates the presentation background, can be overridden
// to return a background image or gradient
@mixin bodyBackground() {
background: $backgroundColor;
}
doc/pub/Linalg/html/reveal.js/css/theme/template/theme.scss
+171
View File
@@ -0,0 +1,171 @@
// Base theme template for reveal.js
/*********************************************
* GLOBAL STYLES
*********************************************/
body {
@include bodyBackground();
background-color: $backgroundColor;
}
.reveal {
font-family: $mainFont;
font-size: $mainFontSize;
font-weight: normal;
letter-spacing: -0.02em;
color: $mainColor;
}
::selection {
color: $selectionColor;
background: $selectionBackgroundColor;
text-shadow: none;
}
/*********************************************
* HEADERS
*********************************************/
.reveal h1,
.reveal h2,
.reveal h3,
.reveal h4,
.reveal h5,
.reveal h6 {
margin: $headingMargin;
color: $headingColor;
font-family: $headingFont;
line-height: $headingLineHeight;
letter-spacing: $headingLetterSpacing;
text-transform: $headingTextTransform;
text-shadow: $headingTextShadow;
}
.reveal h1 {
line-height: 1.2em; /* added by hpl */
text-shadow: $heading1TextShadow;
}
/*********************************************
* LINKS
*********************************************/
.reveal a:not(.image) {
color: $linkColor;
text-decoration: none;
-webkit-transition: color .15s ease;
-moz-transition: color .15s ease;
-ms-transition: color .15s ease;
-o-transition: color .15s ease;
transition: color .15s ease;
}
.reveal a:not(.image):hover {
color: $linkColorHover;
text-shadow: none;
border: none;
}
.reveal .roll span:after {
color: #fff;
background: darken( $linkColor, 15% );
}
/*********************************************
* IMAGES
*********************************************/
.reveal section img {
margin: 15px 0px;
background: rgba(255,255,255,0.12);
border: 4px solid $mainColor;
box-shadow: 0 0 10px rgba(0, 0, 0, 0.15);
-webkit-transition: all .2s linear;
-moz-transition: all .2s linear;
-ms-transition: all .2s linear;
-o-transition: all .2s linear;
transition: all .2s linear;
}
.reveal a:hover img {
background: rgba(255,255,255,0.2);
border-color: $linkColor;
box-shadow: 0 0 20px rgba(0, 0, 0, 0.55);
}
/*********************************************
* NAVIGATION CONTROLS
*********************************************/
.reveal .controls div.navigate-left,
.reveal .controls div.navigate-left.enabled {
border-right-color: $linkColor;
}
.reveal .controls div.navigate-right,
.reveal .controls div.navigate-right.enabled {
border-left-color: $linkColor;
}
.reveal .controls div.navigate-up,
.reveal .controls div.navigate-up.enabled {
border-bottom-color: $linkColor;
}
.reveal .controls div.navigate-down,
.reveal .controls div.navigate-down.enabled {
border-top-color: $linkColor;
}
.reveal .controls div.navigate-left.enabled:hover {
border-right-color: $linkColorHover;
}
.reveal .controls div.navigate-right.enabled:hover {
border-left-color: $linkColorHover;
}
.reveal .controls div.navigate-up.enabled:hover {
border-bottom-color: $linkColorHover;
}
.reveal .controls div.navigate-down.enabled:hover {
border-top-color: $linkColorHover;
}
/*********************************************
* PROGRESS BAR
*********************************************/
.reveal .progress {
background: rgba(0,0,0,0.2);
}
.reveal .progress span {
background: $linkColor;
-webkit-transition: width 800ms cubic-bezier(0.260, 0.860, 0.440, 0.985);
-moz-transition: width 800ms cubic-bezier(0.260, 0.860, 0.440, 0.985);
-ms-transition: width 800ms cubic-bezier(0.260, 0.860, 0.440, 0.985);
-o-transition: width 800ms cubic-bezier(0.260, 0.860, 0.440, 0.985);
transition: width 800ms cubic-bezier(0.260, 0.860, 0.440, 0.985);
}
/*********************************************
* SLIDE NUMBER
*********************************************/
.reveal .slide-number {
color: $linkColor;
}
doc/pub/Linalg/html/reveal.js/demo.html
+410
View File
@@ -0,0 +1,410 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>reveal.js – The HTML Presentation Framework</title>
<meta name="description" content="A framework for easily creating beautiful presentations using HTML">
<meta name="author" content="Hakim El Hattab">
<meta name="apple-mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent">
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no">
<link rel="stylesheet" href="css/reveal.css">
<link rel="stylesheet" href="css/theme/black.css" id="theme">
<!-- Theme used for syntax highlighting of code -->
<link rel="stylesheet" href="lib/css/zenburn.css">
<!-- Printing and PDF exports -->
<script>
var link = document.createElement( 'link' );
link.rel = 'stylesheet';
link.type = 'text/css';
link.href = window.location.search.match( /print-pdf/gi ) ? 'css/print/pdf.css' : 'css/print/paper.css';
document.getElementsByTagName( 'head' )[0].appendChild( link );
</script>
<!--[if lt IE 9]>
<script src="lib/js/html5shiv.js"></script>
<![endif]-->
</head>
<body>
<div class="reveal">
<!-- Any section element inside of this container is displayed as a slide -->
<div class="slides">
<section>
<h1>Reveal.js</h1>
<h3>The HTML Presentation Framework</h3>
<p>
<small>Created by <a href="http://hakim.se">Hakim El Hattab</a> and <a href="https://github.com/hakimel/reveal.js/graphs/contributors">contributors</a></small>
</p>
</section>
<section>
<h2>Hello There</h2>
<p>
reveal.js enables you to create beautiful interactive slide decks using HTML. This presentation will show you examples of what it can do.
</p>
</section>
<!-- Example of nested vertical slides -->
<section>
<section>
<h2>Vertical Slides</h2>
<p>Slides can be nested inside of each other.</p>
<p>Use the <em>Space</em> key to navigate through all slides.</p>
<br>
<a href="#" class="navigate-down">
<img width="178" height="238" data-src="https://s3.amazonaws.com/hakim-static/reveal-js/arrow.png" alt="Down arrow">
</a>
</section>
<section>
<h2>Basement Level 1</h2>
<p>Nested slides are useful for adding additional detail underneath a high level horizontal slide.</p>
</section>
<section>
<h2>Basement Level 2</h2>
<p>That's it, time to go back up.</p>
<br>
<a href="#/2">
<img width="178" height="238" data-src="https://s3.amazonaws.com/hakim-static/reveal-js/arrow.png" alt="Up arrow" style="transform: rotate(180deg); -webkit-transform: rotate(180deg);">
</a>
</section>
</section>
<section>
<h2>Slides</h2>
<p>
Not a coder? Not a problem. There's a fully-featured visual editor for authoring these, try it out at <a href="https://slides.com" target="_blank">https://slides.com</a>.
</p>
</section>
<section>
<h2>Point of View</h2>
<p>
Press <strong>ESC</strong> to enter the slide overview.
</p>
<p>
Hold down alt and click on any element to zoom in on it using <a href="http://lab.hakim.se/zoom-js">zoom.js</a>. Alt + click anywhere to zoom back out.
</p>
</section>
<section>
<h2>Touch Optimized</h2>
<p>
Presentations look great on touch devices, like mobile phones and tablets. Simply swipe through your slides.
</p>
</section>
<section data-markdown>
<script type="text/template">
## Markdown support
Write content using inline or external Markdown.
Instructions and more info available in the [readme](https://github.com/hakimel/reveal.js#markdown).
```
<section data-markdown>
## Markdown support
Write content using inline or external Markdown.
Instructions and more info available in the [readme](https://github.com/hakimel/reveal.js#markdown).
</section>
```
</script>
</section>
<section>
<section id="fragments">
<h2>Fragments</h2>
<p>Hit the next arrow...</p>
<p class="fragment">... to step through ...</p>
<p><span class="fragment">... a</span> <span class="fragment">fragmented</span> <span class="fragment">slide.</span></p>
<aside class="notes">
This slide has fragments which are also stepped through in the notes window.
</aside>
</section>
<section>
<h2>Fragment Styles</h2>
<p>There's different types of fragments, like:</p>
<p class="fragment grow">grow</p>
<p class="fragment shrink">shrink</p>
<p class="fragment fade-out">fade-out</p>
<p class="fragment fade-up">fade-up (also down, left and right!)</p>
<p class="fragment current-visible">current-visible</p>
<p>Highlight <span class="fragment highlight-red">red</span> <span class="fragment highlight-blue">blue</span> <span class="fragment highlight-green">green</span></p>
</section>
</section>
<section id="transitions">
<h2>Transition Styles</h2>
<p>
You can select from different transitions, like: <br>
<a href="?transition=none#/transitions">None</a> -
<a href="?transition=fade#/transitions">Fade</a> -
<a href="?transition=slide#/transitions">Slide</a> -
<a href="?transition=convex#/transitions">Convex</a> -
<a href="?transition=concave#/transitions">Concave</a> -
<a href="?transition=zoom#/transitions">Zoom</a>
</p>
</section>
<section id="themes">
<h2>Themes</h2>
<p>
reveal.js comes with a few themes built in: <br>
<!-- Hacks to swap themes after the page has loaded. Not flexible and only intended for the reveal.js demo deck. -->
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/black.css'); return false;">Black (default)</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/white.css'); return false;">White</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/league.css'); return false;">League</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/sky.css'); return false;">Sky</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/beige.css'); return false;">Beige</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/simple.css'); return false;">Simple</a> <br>
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/serif.css'); return false;">Serif</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/blood.css'); return false;">Blood</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/night.css'); return false;">Night</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/moon.css'); return false;">Moon</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/solarized.css'); return false;">Solarized</a>
</p>
</section>
<section>
<section data-background="#dddddd">
<h2>Slide Backgrounds</h2>
<p>
Set <code>data-background="#dddddd"</code> on a slide to change the background color. All CSS color formats are supported.
</p>
<a href="#" class="navigate-down">
<img width="178" height="238" data-src="https://s3.amazonaws.com/hakim-static/reveal-js/arrow.png" alt="Down arrow">
</a>
</section>
<section data-background="https://s3.amazonaws.com/hakim-static/reveal-js/image-placeholder.png">
<h2>Image Backgrounds</h2>
<pre><code class="hljs">&lt;section data-background="image.png"&gt;</code></pre>
</section>
<section data-background="https://s3.amazonaws.com/hakim-static/reveal-js/image-placeholder.png" data-background-repeat="repeat" data-background-size="100px">
<h2>Tiled Backgrounds</h2>
<pre><code class="hljs" style="word-wrap: break-word;">&lt;section data-background="image.png" data-background-repeat="repeat" data-background-size="100px"&gt;</code></pre>
</section>
<section data-background-video="https://s3.amazonaws.com/static.slid.es/site/homepage/v1/homepage-video-editor.mp4,https://s3.amazonaws.com/static.slid.es/site/homepage/v1/homepage-video-editor.webm" data-background-color="#000000">
<div style="background-color: rgba(0, 0, 0, 0.9); color: #fff; padding: 20px;">
<h2>Video Backgrounds</h2>
<pre><code class="hljs" style="word-wrap: break-word;">&lt;section data-background-video="video.mp4,video.webm"&gt;</code></pre>
</div>
</section>
<section data-background="http://i.giphy.com/90F8aUepslB84.gif">
<h2>... and GIFs!</h2>
</section>
</section>
<section data-transition="slide" data-background="#4d7e65" data-background-transition="zoom">
<h2>Background Transitions</h2>
<p>
Different background transitions are available via the backgroundTransition option. This one's called "zoom".
</p>
<pre><code class="hljs">Reveal.configure({ backgroundTransition: 'zoom' })</code></pre>
</section>
<section data-transition="slide" data-background="#b5533c" data-background-transition="zoom">
<h2>Background Transitions</h2>
<p>
You can override background transitions per-slide.
</p>
<pre><code class="hljs" style="word-wrap: break-word;">&lt;section data-background-transition="zoom"&gt;</code></pre>
</section>
<section>
<h2>Pretty Code</h2>
<pre><code class="hljs" data-trim contenteditable>
function linkify( selector ) {
if( supports3DTransforms ) {
var nodes = document.querySelectorAll( selector );
for( var i = 0, len = nodes.length; i &lt; len; i++ ) {
var node = nodes[i];
if( !node.className ) {
node.className += ' roll';
}
}
}
}
</code></pre>
<p>Code syntax highlighting courtesy of <a href="http://softwaremaniacs.org/soft/highlight/en/description/">highlight.js</a>.</p>
</section>
<section>
<h2>Marvelous List</h2>
<ul>
<li>No order here</li>
<li>Or here</li>
<li>Or here</li>
<li>Or here</li>
</ul>
</section>
<section>
<h2>Fantastic Ordered List</h2>
<ol>
<li>One is smaller than...</li>
<li>Two is smaller than...</li>
<li>Three!</li>
</ol>
</section>
<section>
<h2>Tabular Tables</h2>
<table>
<thead>
<tr>
<th>Item</th>
<th>Value</th>
<th>Quantity</th>
</tr>
</thead>
<tbody>
<tr>
<td>Apples</td>
<td>$1</td>
<td>7</td>
</tr>
<tr>
<td>Lemonade</td>
<td>$2</td>
<td>18</td>
</tr>
<tr>
<td>Bread</td>
<td>$3</td>
<td>2</td>
</tr>
</tbody>
</table>
</section>
<section>
<h2>Clever Quotes</h2>
<p>
These guys come in two forms, inline: <q cite="http://searchservervirtualization.techtarget.com/definition/Our-Favorite-Technology-Quotations">The nice thing about standards is that there are so many to choose from</q> and block:
</p>
<blockquote cite="http://searchservervirtualization.techtarget.com/definition/Our-Favorite-Technology-Quotations">
&ldquo;For years there has been a theory that millions of monkeys typing at random on millions of typewriters would
reproduce the entire works of Shakespeare. The Internet has proven this theory to be untrue.&rdquo;
</blockquote>
</section>
<section>
<h2>Intergalactic Interconnections</h2>
<p>
You can link between slides internally,
<a href="#/2/3">like this</a>.
</p>
</section>
<section>
<h2>Speaker View</h2>
<p>There's a <a href="https://github.com/hakimel/reveal.js#speaker-notes">speaker view</a>. It includes a timer, preview of the upcoming slide as well as your speaker notes.</p>
<p>Press the <em>S</em> key to try it out.</p>
<aside class="notes">
Oh hey, these are some notes. They'll be hidden in your presentation, but you can see them if you open the speaker notes window (hit 's' on your keyboard).
</aside>
</section>
<section>
<h2>Export to PDF</h2>
<p>Presentations can be <a href="https://github.com/hakimel/reveal.js#pdf-export">exported to PDF</a>, here's an example:</p>
<iframe data-src="https://www.slideshare.net/slideshow/embed_code/42840540" width="445" height="355" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" style="border:3px solid #666; margin-bottom:5px; max-width: 100%;" allowfullscreen> </iframe>
</section>
<section>
<h2>Global State</h2>
<p>
Set <code>data-state="something"</code> on a slide and <code>"something"</code>
will be added as a class to the document element when the slide is open. This lets you
apply broader style changes, like switching the page background.
</p>
</section>
<section data-state="customevent">
<h2>State Events</h2>
<p>
Additionally custom events can be triggered on a per slide basis by binding to the <code>data-state</code> name.
</p>
<pre><code class="javascript" data-trim contenteditable style="font-size: 18px;">
Reveal.addEventListener( 'customevent', function() {
console.log( '"customevent" has fired' );
} );
</code></pre>
</section>
<section>
<h2>Take a Moment</h2>
<p>
Press B or . on your keyboard to pause the presentation. This is helpful when you're on stage and want to take distracting slides off the screen.
</p>
</section>
<section>
<h2>Much more</h2>
<ul>
<li>Right-to-left support</li>
<li><a href="https://github.com/hakimel/reveal.js#api">Extensive JavaScript API</a></li>
<li><a href="https://github.com/hakimel/reveal.js#auto-sliding">Auto-progression</a></li>
<li><a href="https://github.com/hakimel/reveal.js#parallax-background">Parallax backgrounds</a></li>
<li><a href="https://github.com/hakimel/reveal.js#keyboard-bindings">Custom keyboard bindings</a></li>
</ul>
</section>
<section style="text-align: left;">
<h1>THE END</h1>
<p>
- <a href="https://slides.com">Try the online editor</a> <br>
- <a href="https://github.com/hakimel/reveal.js">Source code &amp; documentation</a>
</p>
</section>
</div>
</div>
<script src="lib/js/head.min.js"></script>
<script src="js/reveal.js"></script>
<script>
// More info https://github.com/hakimel/reveal.js#configuration
Reveal.initialize({
controls: true,
progress: true,
history: true,
center: true,
transition: 'slide', // none/fade/slide/convex/concave/zoom
// More info https://github.com/hakimel/reveal.js#dependencies
dependencies: [
{ src: 'lib/js/classList.js', condition: function() { return !document.body.classList; } },
{ src: 'plugin/markdown/marked.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } },
{ src: 'plugin/markdown/markdown.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } },
{ src: 'plugin/highlight/highlight.js', async: true, callback: function() { hljs.initHighlightingOnLoad(); } },
{ src: 'plugin/search/search.js', async: true },
{ src: 'plugin/zoom-js/zoom.js', async: true },
{ src: 'plugin/notes/notes.js', async: true }
]
});
</script>
</body>
</html>
doc/pub/Linalg/html/reveal.js/plugin/multiplex/package.json
+19
View File
@@ -0,0 +1,19 @@
{
"name": "reveal-js-multiplex",
"version": "1.0.0",
"description": "reveal.js multiplex server",
"homepage": "http://revealjs.com",
"scripts": {
"start": "node index.js"
},
"engines": {
"node": "~4.1.1"
},
"dependencies": {
"express": "~4.13.3",
"grunt-cli": "~0.1.13",
"mustache": "~2.2.1",
"socket.io": "~1.3.7"
},
"license": "MIT"
}
doc/pub/Linalg/html/reveal.js/test/simple.md
+12
View File
@@ -0,0 +1,12 @@
## Slide 1.1
```js
var a = 1;
```
## Slide 1.2
## Slide 2
doc/pub/Linalg/html/reveal.js/test/test-markdown-external.html
+36
View File
@@ -0,0 +1,36 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>reveal.js - Test Markdown</title>
<link rel="stylesheet" href="../css/reveal.css">
<link rel="stylesheet" href="qunit-1.12.0.css">
</head>
<body style="overflow: auto;">
<div id="qunit"></div>
<div id="qunit-fixture"></div>
<div class="reveal" style="display: none;">
<div class="slides">
<section data-markdown="simple.md" data-separator="^\n\n\n" data-separator-vertical="^\n\n"></section>
</div>
</div>
<script src="../lib/js/head.min.js"></script>
<script src="../js/reveal.js"></script>
<script src="../plugin/highlight/highlight.js"></script>
<script src="../plugin/markdown/marked.js"></script>
<script src="../plugin/markdown/markdown.js"></script>
<script src="qunit-1.12.0.js"></script>
<script src="test-markdown-external.js"></script>
</body>
</html>
doc/pub/Linalg/html/reveal.js/test/test-markdown-external.js
+24
View File
@@ -0,0 +1,24 @@
Reveal.addEventListener( 'ready', function() {
QUnit.module( 'Markdown' );
test( 'Vertical separator', function() {
strictEqual( document.querySelectorAll( '.reveal .slides>section>section' ).length, 2, 'found two slides' );
});
test( 'Horizontal separator', function() {
strictEqual( document.querySelectorAll( '.reveal .slides>section' ).length, 2, 'found two slides' );
});
test( 'Language highlighter', function() {
strictEqual( document.querySelectorAll( '.hljs-keyword' ).length, 1, 'got rendered highlight tag.' );
strictEqual( document.querySelector( '.hljs-keyword' ).innerHTML, 'var', 'the same keyword: var.' );
});
} );
Reveal.initialize();
doc/pub/Linalg/html/reveal.js/test/test-markdown-options.html
+41
View File
@@ -0,0 +1,41 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>reveal.js - Test Markdown Options</title>
<link rel="stylesheet" href="../css/reveal.css">
<link rel="stylesheet" href="qunit-1.12.0.css">
</head>
<body style="overflow: auto;">
<div id="qunit"></div>
<div id="qunit-fixture"></div>
<div class="reveal" style="display: none;">
<div class="slides">
<section data-markdown>
<script type="text/template">
## Testing Markdown Options
This "slide" should contain 'smart' quotes.
</script>
</section>
</div>
</div>
<script src="../lib/js/head.min.js"></script>
<script src="../js/reveal.js"></script>
<script src="qunit-1.12.0.js"></script>
<script src="test-markdown-options.js"></script>
</body>
</html>
doc/pub/Linalg/html/reveal.js/test/test-markdown-options.js
+26
View File
@@ -0,0 +1,26 @@
Reveal.addEventListener( 'ready', function() {
QUnit.module( 'Markdown' );
test( 'Options are set', function() {
strictEqual( marked.defaults.smartypants, true );
});
test( 'Smart quotes are activated', function() {
var text = document.querySelector( '.reveal .slides>section>p' ).textContent;
strictEqual( /['"]/.test( text ), false );
strictEqual( /[“”‘’]/.test( text ), true );
});
} );
Reveal.initialize({
dependencies: [
{ src: '../plugin/markdown/marked.js' },
{ src: '../plugin/markdown/markdown.js' },
],
markdown: {
smartypants: true
}
});
doc/pub/Linalg/ipynb/.ipynb_checkpoints/Linalg-checkpoint.ipynb
+2029
View File
File diff suppressed because it is too large Load Diff
doc/pub/LogReg/ipynb/.ipynb_checkpoints/LogReg-checkpoint.ipynb
+502
View File
@@ -0,0 +1,502 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<!-- dom:TITLE: Data Analysis and Machine Learning: Logistic Regression -->\n",
"# Data Analysis and Machine Learning: Logistic Regression\n",
"<!-- dom:AUTHOR: Morten Hjorth-Jensen at Department of Physics, University of Oslo & Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University -->\n",
"<!-- Author: --> \n",
"**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n",
"\n",
"Date: **Sep 26, 2018**\n",
"\n",
"Copyright 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n",
"\n",
"\n",
"\n",
"\n",
"<!-- !split -->\n",
"## Logistic Regression\n",
"\n",
"In linear regression our main interest was centered on learning the\n",
"coefficients of a functional fit (say a polynomial) in order to be\n",
"able to predict the response of a continuous variable on some unseen\n",
"data. The fit to the continuous variable $y_i$ is based on some\n",
"independent variables $\\hat{x}_i$. Linear regression resulted in\n",
"analytical expressions (in terms of matrices to invert) for several\n",
"quantities, ranging from the variance and thereby the confidence\n",
"intervals of the parameters $\\hat{\\beta}$ to the mean squared\n",
"error. If we can invert the product of the design matrices, linear\n",
"regression gives then a simple recipe for fitting our data.\n",
"\n",
"\n",
"Classification problems, however, are concerned with outcomes taking\n",
"the form of discrete variables (i.e. categories). We may for example,\n",
"on the basis of DNA sequencing for a number of patients, like to find\n",
"out which mutations are important for a certain disease; or based on\n",
"scans of various patients' brains, figure out if there is a tumor or\n",
"not; or given a specific physical system, we'd like to identify its\n",
"state, say whether it is an ordered or disordered system (typical\n",
"situation in solid state physics); or classify the status of a\n",
"patient, whether she/he has a stroke or not and many other similar\n",
"situations.\n",
"\n",
"The most common situation we encounter when we apply logistic\n",
"regression is that of two possible outcomes, normally denoted as a\n",
"binary outcome, true or false, positive or negative, success or\n",
"failure etc.\n",
"\n",
"## Optimization and Deep learning\n",
"\n",
"Logistic regression will also serve as our stepping stone towards neural\n",
"network algorithms and supervised deep learning. For logistic\n",
"learning, the minimization of the cost function leads to a non-linear\n",
"equation in the parameters $\\hat{\\beta}$. The optmization of the problem calls therefore for minimization algorithms. This forms the bottle neck of all machine learning algorithms, namely how to find reliable minima of a multi-variable function. This leads us to the family of gradient descent methods. The latter are the working horses of basically all modern machine learning algorithms. \n",
"\n",
"We note also that many of the topics discussed here \n",
"regression are also commonly used in modern supervised Deep Learning\n",
"models, as we will see later.\n",
"\n",
"\n",
"<!-- !split -->\n",
"## Basics\n",
"\n",
"We consider the case where the dependent variables, also called the\n",
"responses or the outcomes, $y_i$ are discrete and only take values\n",
"from $k=0,\\dots,K-1$ (i.e. $K$ classes).\n",
"\n",
"The goal is to predict the\n",
"output classes from the design matrix $\\hat{X}\\in\\mathbb{R}^{n\\times p}$\n",
"made of $n$ samples, each of which carries $p$ features or predictors. The\n",
"primary goal is to identify the classes to which new unseen samples\n",
"belong.\n",
"\n",
"Let us specialize to the case of two classes only, with outputs $y_i=0$ and $y_i=1$. Our outcomes could represent the status of a credit card user who could default or not on her/his credit card debt. That is"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"y_i = \\begin{bmatrix} 0 & \\mathrm{no}\\\\ 1 & \\mathrm{yes} \\end{bmatrix}.\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Linear classifier\n",
"\n",
"Before moving to the logistic model, let us try to use our linear regression model to classify these two outcomes. We could for example fit a linear model to the default case if $y_i > 0.5$ and the no default case $y_i \\leq 0.5$. \n",
"\n",
"We would then have our \n",
"weighted linear combination, namely"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<!-- Equation labels as ordinary links -->\n",
"<div id=\"_auto1\"></div>\n",
"\n",
"$$\n",
"\\begin{equation}\n",
"\\hat{y} = \\hat{X}^T\\hat{\\beta} + \\hat{\\epsilon},\n",
"\\label{_auto1} \\tag{1}\n",
"\\end{equation}\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"where $\\hat{y}$ is a vector representing the possible outcomes, $\\hat{X}$ is our\n",
"$n\\times p$ design matrix and $\\hat{\\beta}$ represents our estimators/predictors.\n",
"\n",
"## Some selected properties\n",
"\n",
"The main problem with our function is that it \n",
"takes values on the entire real axis. In the case of\n",
"logistic regression, however, the labels $y_i$ are discrete\n",
"variables. \n",
"\n",
"One simple way to get a discrete output is to have sign\n",
"functions that map the output of a linear regressor to values $\\{0,1\\}$,\n",
"$f(s_i)=sign(s_i)=1$ if $s_i\\ge 0$ and 0 if otherwise. \n",
"We will encounter this model in our first demonstration of neural networks. Historically it is called the \"perceptron\" model in the machine learning\n",
"literature. This model is extremely simple. However, in many cases it is more\n",
"favorable to use a ``soft\" classifier that outputs\n",
"the probability of a given category. This leads us to the logistic function.\n",
"\n",
"The code for plotting the perceptron can be seen here. This si nothing but the standard [Heaviside step function](https://en.wikipedia.org/wiki/Heaviside_step_function)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## The logistic function\n",
"\n",
"The perceptron is an example of a ``hard classification\" model. We\n",
"will encounter this model when we discuss neural networks as\n",
"well. Each datapoint is deterministically assigned to a category (i.e\n",
"$y_i=0$ or $y_i=1$). In many cases, it is favorable to have a \"soft\"\n",
"classifier that outputs the probability of a given category rather\n",
"than a single value. For example, given $x_i$, the classifier\n",
"outputs the probability of being in a category $k$. Logistic regression\n",
"is the most common example of a so-called soft classifier. In logistic\n",
"regression, the probability that a data point $x_i$\n",
"belongs to a category $y_i=\\{0,1\\}$ is given by the so-called logit function (or Sigmoid) which is meant to represent the likelihood for a given event,"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"p(t) = \\frac{1}{1+\\mathrm \\exp{-t}}=\\frac{\\exp{t}}{1+\\mathrm \\exp{t}}.\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that $1-p(t)= p(-t)$.\n",
"The following code plots the logistic function."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Two parameters\n",
"\n",
"We assume now that we have two classes with $y_i$ either $0$ or $1$. Furthermore we assume also that we have only two parameters $\\beta$ in our fitting of the Sigmoid function, that is we define probabilities"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"\\begin{align*}\n",
"p(y_i=1|x_i,\\hat{\\beta}) &= \\frac{\\exp{(\\beta_0+\\beta_1x_i)}}{1+\\exp{(\\beta_0+\\beta_1x_i)}},\\nonumber\\\\\n",
"p(y_i=0|x_i,\\hat{\\beta}) &= 1 - p(y_i=1|x_i,\\hat{\\beta}),\n",
"\\end{align*}\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"where $\\hat{\\beta}$ are the weights we wish to extract from data, in our case $\\beta_0$ and $\\beta_1$. \n",
"\n",
"Note that we used"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"p(y_i=0\\vert x_i, \\hat{\\beta}) = 1-p(y_i=1\\vert x_i, \\hat{\\beta}).\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<!-- !split -->\n",
"## Maximum likelihood\n",
"\n",
"In order to define the total likelihood for all possible outcomes from a \n",
"dataset $\\mathcal{D}=\\{(y_i,x_i)\\}$, with the binary labels\n",
"$y_i\\in\\{0,1\\}$ and where the data points are drawn independently, we use the so-called [Maximum Likelihood Estimation](https://en.wikipedia.org/wiki/Maximum_likelihood_estimation) (MLE) principle. \n",
"We aim thus at maximizing \n",
"the probability of seeing the observed data. We can then approximate the \n",
"likelihood in terms of the product of the individual probabilities of a specific outcome $y_i$, that is"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"\\begin{align*}\n",
"P(\\mathcal{D}|\\hat{\\beta})& = \\prod_{i=1}^n \\left[p(y_i=1|x_i,\\hat{\\beta})\\right]^{y_i}\\left[1-p(y_i=1|x_i,\\hat{\\beta}))\\right]^{1-y_i}\\nonumber \\\\\n",
"\\end{align*}\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"from which we obtain the log-likelihood and our **cost/loss** function"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"\\mathcal{C}(\\hat{\\beta}) = \\sum_{i=1}^n \\left( y_i\\log{p(y_i=1|x_i,\\hat{\\beta})} + (1-y_i)\\log\\left[1-p(y_i=1|x_i,\\hat{\\beta}))\\right]\\right).\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## The cost function rewritten\n",
"\n",
"Reordering the logarithms, we can rewrite the **cost/loss** function as"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"\\mathcal{C}(\\hat{\\beta}) = \\sum_{i=1}^n \\left(y_i(\\beta_0+\\beta_1x_i) -\\log{(1+\\exp{(\\beta_0+\\beta_1x_i)})}\\right).\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The maximum likelihood estimator is defined as the set of parameters that maximize the log-likelihood where we maximize with respect to $\\beta$.\n",
"Since the cost (error) function is just the negative log-likelihood, for logistic regression we have that"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"\\mathcal{C}(\\hat{\\beta})=-\\sum_{i=1}^n \\left(y_i(\\beta_0+\\beta_1x_i) -\\log{(1+\\exp{(\\beta_0+\\beta_1x_i)})}\\right).\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This equation is known in statistics as the **cross entropy**. Finally, we note that just as in linear regression, \n",
"in practice we often supplement the cross-entropy with additional regularization terms, usually $L_1$ and $L_2$ regularization as we did for Ridge and Lasso regression.\n",
"\n",
"## Minimizing the cross entropy\n",
"\n",
"The cross entropy is a convex function of the weights $\\hat{\\beta}$ and,\n",
"therefore, any local minimizer is a global minimizer. \n",
"\n",
"\n",
"Minimizing this\n",
"cost function with respect to the two parameters $\\beta_0$ and $\\beta_1$ we obtain"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"\\frac{\\partial \\mathcal{C}(\\hat{\\beta})}{\\partial \\beta_0} = -\\sum_{i=1}^n \\left(y_i -\\frac{\\exp{(\\beta_0+\\beta_1x_i)}}{1+\\exp{(\\beta_0+\\beta_1x_i)}}\\right),\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"and"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"\\frac{\\partial \\mathcal{C}(\\hat{\\beta})}{\\partial \\beta_1} = -\\sum_{i=1}^n \\left(y_ix_i -x_i\\frac{\\exp{(\\beta_0+\\beta_1x_i)}}{1+\\exp{(\\beta_0+\\beta_1x_i)}}\\right).\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## A more compact expression\n",
"\n",
"Let us now define a vector $\\hat{y}$ with $n$ elements $y_i$, an\n",
"$n\\times p$ matrix $\\hat{X}$ which contains the $x_i$ values and a\n",
"vector $\\hat{p}$ of fitted probabilities $p(y_i\\vert x_i,\\hat{\\beta})$. We can rewrite in a more compact form the first\n",
"derivative of cost function as"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"\\frac{\\partial \\mathcal{C}(\\hat{\\beta})}{\\partial \\hat{\\beta}} = -\\hat{X}^T\\left(\\hat{y}-\\hat{p}\\right).\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If we in addition define a diagonal matrix $\\hat{W}$ with elements \n",
"$p(y_i\\vert x_i,\\hat{\\beta})(1-p(y_i\\vert x_i,\\hat{\\beta})$, we can obtain a compact expression of the second derivative as"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"\\frac{\\partial^2 \\mathcal{C}(\\hat{\\beta})}{\\partial \\hat{\\beta}\\partial \\hat{\\beta}^T} = \\hat{X}^T\\hat{W}\\hat{X}.\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Extending to more predictors\n",
"\n",
"## Including more classes\n",
"\n",
"## Optimizing the cost function\n",
"\n",
"Newton's method and gradient descent methods\n",
"\n",
"\n",
"\n",
"## A **scikit-learn** example"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"%matplotlib inline\n",
"\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from sklearn import datasets\n",
"iris = datasets.load_iris()\n",
"list(iris.keys())\n",
"['data', 'target_names', 'feature_names', 'target', 'DESCR']\n",
"X = iris[\"data\"][:, 3:] # petal width\n",
"y = (iris[\"target\"] == 2).astype(np.int) # 1 if Iris-Virginica, else 0\n",
"\n",
"from sklearn.linear_model import LogisticRegression\n",
"log_reg = LogisticRegression()\n",
"log_reg.fit(X, y)\n",
"\n",
"X_new = np.linspace(0, 3, 1000).reshape(-1, 1)\n",
"y_proba = log_reg.predict_proba(X_new)\n",
"plt.plot(X_new, y_proba[:, 1], \"g-\", label=\"Iris-Virginica\")\n",
"plt.plot(X_new, y_proba[:, 0], \"b--\", label=\"Not Iris-Virginica\")\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## A simple classification problem"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"from sklearn import datasets, linear_model\n",
"import matplotlib.pyplot as plt\n",
"\n",
"\n",
"def generate_data():\n",
" np.random.seed(0)\n",
" X, y = datasets.make_moons(200, noise=0.20)\n",
" return X, y\n",
"\n",
"\n",
"def visualize(X, y, clf):\n",
" # plt.scatter(X[:, 0], X[:, 1], s=40, c=y, cmap=plt.cm.Spectral)\n",
" # plt.show()\n",
" plot_decision_boundary(lambda x: clf.predict(x), X, y)\n",
" plt.title(\"Logistic Regression\")\n",
"\n",
"\n",
"def plot_decision_boundary(pred_func, X, y):\n",
" # Set min and max values and give it some padding\n",
" x_min, x_max = X[:, 0].min() - .5, X[:, 0].max() + .5\n",
" y_min, y_max = X[:, 1].min() - .5, X[:, 1].max() + .5\n",
" h = 0.01\n",
" # Generate a grid of points with distance h between them\n",
" xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))\n",
" # Predict the function value for the whole gid\n",
" Z = pred_func(np.c_[xx.ravel(), yy.ravel()])\n",
" Z = Z.reshape(xx.shape)\n",
" # Plot the contour and training examples\n",
" plt.contourf(xx, yy, Z, cmap=plt.cm.Spectral)\n",
" plt.scatter(X[:, 0], X[:, 1], c=y, cmap=plt.cm.Spectral)\n",
" plt.show()\n",
"\n",
"\n",
"def classify(X, y):\n",
" clf = linear_model.LogisticRegressionCV()\n",
" clf.fit(X, y)\n",
" return clf\n",
"\n",
"\n",
"def main():\n",
" X, y = generate_data()\n",
" # visualize(X, y)\n",
" clf = classify(X, y)\n",
" visualize(X, y, clf)\n",
"\n",
"\n",
"if __name__ == \"__main__\":\n",
" main()"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.0"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
doc/pub/NeuralNet/html/reveal.js/css/theme/template/mixins.scss
+29
View File
@@ -0,0 +1,29 @@
@mixin vertical-gradient( $top, $bottom ) {
background: $top;
background: -moz-linear-gradient( top, $top 0%, $bottom 100% );
background: -webkit-gradient( linear, left top, left bottom, color-stop(0%,$top), color-stop(100%,$bottom) );
background: -webkit-linear-gradient( top, $top 0%, $bottom 100% );
background: -o-linear-gradient( top, $top 0%, $bottom 100% );
background: -ms-linear-gradient( top, $top 0%, $bottom 100% );
background: linear-gradient( top, $top 0%, $bottom 100% );
}
@mixin horizontal-gradient( $top, $bottom ) {
background: $top;
background: -moz-linear-gradient( left, $top 0%, $bottom 100% );
background: -webkit-gradient( linear, left top, right top, color-stop(0%,$top), color-stop(100%,$bottom) );
background: -webkit-linear-gradient( left, $top 0%, $bottom 100% );
background: -o-linear-gradient( left, $top 0%, $bottom 100% );
background: -ms-linear-gradient( left, $top 0%, $bottom 100% );
background: linear-gradient( left, $top 0%, $bottom 100% );
}
@mixin radial-gradient( $outer, $inner, $type: circle ) {
background: $outer;
background: -moz-radial-gradient( center, $type cover, $inner 0%, $outer 100% );
background: -webkit-gradient( radial, center center, 0px, center center, 100%, color-stop(0%,$inner), color-stop(100%,$outer) );
background: -webkit-radial-gradient( center, $type cover, $inner 0%, $outer 100% );
background: -o-radial-gradient( center, $type cover, $inner 0%, $outer 100% );
background: -ms-radial-gradient( center, $type cover, $inner 0%, $outer 100% );
background: radial-gradient( center, $type cover, $inner 0%, $outer 100% );
}
doc/pub/NeuralNet/html/reveal.js/css/theme/template/settings.scss
+34
View File
@@ -0,0 +1,34 @@
// Base settings for all themes that can optionally be
// overridden by the super-theme
// Background of the presentation
$backgroundColor: #2b2b2b;
// Primary/body text
$mainFont: 'Lato', sans-serif;
$mainFontSize: 30px; /* changed (by hpl) from 36px; */
$mainColor: #eee;
// Headings
$headingMargin: 0 0 20px 0;
$headingFont: 'League Gothic', Impact, sans-serif;
$headingColor: #eee;
$headingLineHeight: 0.9em;
$headingLetterSpacing: 0.02em;
$headingTextTransform: none; /* changed (by hpl) from uppercase; */
$headingTextShadow: 0px 0px 6px rgba(0,0,0,0.2);
$heading1TextShadow: $headingTextShadow;
// Links and actions
$linkColor: #13DAEC;
$linkColorHover: lighten( $linkColor, 20% );
// Text selection
$selectionBackgroundColor: #FF5E99;
$selectionColor: #fff;
// Generates the presentation background, can be overridden
// to return a background image or gradient
@mixin bodyBackground() {
background: $backgroundColor;
}
doc/pub/NeuralNet/html/reveal.js/css/theme/template/theme.scss
+171
View File
@@ -0,0 +1,171 @@
// Base theme template for reveal.js
/*********************************************
* GLOBAL STYLES
*********************************************/
body {
@include bodyBackground();
background-color: $backgroundColor;
}
.reveal {
font-family: $mainFont;
font-size: $mainFontSize;
font-weight: normal;
letter-spacing: -0.02em;
color: $mainColor;
}
::selection {
color: $selectionColor;
background: $selectionBackgroundColor;
text-shadow: none;
}
/*********************************************
* HEADERS
*********************************************/
.reveal h1,
.reveal h2,
.reveal h3,
.reveal h4,
.reveal h5,
.reveal h6 {
margin: $headingMargin;
color: $headingColor;
font-family: $headingFont;
line-height: $headingLineHeight;
letter-spacing: $headingLetterSpacing;
text-transform: $headingTextTransform;
text-shadow: $headingTextShadow;
}
.reveal h1 {
line-height: 1.2em; /* added by hpl */
text-shadow: $heading1TextShadow;
}
/*********************************************
* LINKS
*********************************************/
.reveal a:not(.image) {
color: $linkColor;
text-decoration: none;
-webkit-transition: color .15s ease;
-moz-transition: color .15s ease;
-ms-transition: color .15s ease;
-o-transition: color .15s ease;
transition: color .15s ease;
}
.reveal a:not(.image):hover {
color: $linkColorHover;
text-shadow: none;
border: none;
}
.reveal .roll span:after {
color: #fff;
background: darken( $linkColor, 15% );
}
/*********************************************
* IMAGES
*********************************************/
.reveal section img {
margin: 15px 0px;
background: rgba(255,255,255,0.12);
border: 4px solid $mainColor;
box-shadow: 0 0 10px rgba(0, 0, 0, 0.15);
-webkit-transition: all .2s linear;
-moz-transition: all .2s linear;
-ms-transition: all .2s linear;
-o-transition: all .2s linear;
transition: all .2s linear;
}
.reveal a:hover img {
background: rgba(255,255,255,0.2);
border-color: $linkColor;
box-shadow: 0 0 20px rgba(0, 0, 0, 0.55);
}
/*********************************************
* NAVIGATION CONTROLS
*********************************************/
.reveal .controls div.navigate-left,
.reveal .controls div.navigate-left.enabled {
border-right-color: $linkColor;
}
.reveal .controls div.navigate-right,
.reveal .controls div.navigate-right.enabled {
border-left-color: $linkColor;
}
.reveal .controls div.navigate-up,
.reveal .controls div.navigate-up.enabled {
border-bottom-color: $linkColor;
}
.reveal .controls div.navigate-down,
.reveal .controls div.navigate-down.enabled {
border-top-color: $linkColor;
}
.reveal .controls div.navigate-left.enabled:hover {
border-right-color: $linkColorHover;
}
.reveal .controls div.navigate-right.enabled:hover {
border-left-color: $linkColorHover;
}
.reveal .controls div.navigate-up.enabled:hover {
border-bottom-color: $linkColorHover;
}
.reveal .controls div.navigate-down.enabled:hover {
border-top-color: $linkColorHover;
}
/*********************************************
* PROGRESS BAR
*********************************************/
.reveal .progress {
background: rgba(0,0,0,0.2);
}
.reveal .progress span {
background: $linkColor;
-webkit-transition: width 800ms cubic-bezier(0.260, 0.860, 0.440, 0.985);
-moz-transition: width 800ms cubic-bezier(0.260, 0.860, 0.440, 0.985);
-ms-transition: width 800ms cubic-bezier(0.260, 0.860, 0.440, 0.985);
-o-transition: width 800ms cubic-bezier(0.260, 0.860, 0.440, 0.985);
transition: width 800ms cubic-bezier(0.260, 0.860, 0.440, 0.985);
}
/*********************************************
* SLIDE NUMBER
*********************************************/
.reveal .slide-number {
color: $linkColor;
}
doc/pub/NeuralNet/html/reveal.js/demo.html
+410
View File
@@ -0,0 +1,410 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>reveal.js – The HTML Presentation Framework</title>
<meta name="description" content="A framework for easily creating beautiful presentations using HTML">
<meta name="author" content="Hakim El Hattab">
<meta name="apple-mobile-web-app-capable" content="yes">
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent">
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no">
<link rel="stylesheet" href="css/reveal.css">
<link rel="stylesheet" href="css/theme/black.css" id="theme">
<!-- Theme used for syntax highlighting of code -->
<link rel="stylesheet" href="lib/css/zenburn.css">
<!-- Printing and PDF exports -->
<script>
var link = document.createElement( 'link' );
link.rel = 'stylesheet';
link.type = 'text/css';
link.href = window.location.search.match( /print-pdf/gi ) ? 'css/print/pdf.css' : 'css/print/paper.css';
document.getElementsByTagName( 'head' )[0].appendChild( link );
</script>
<!--[if lt IE 9]>
<script src="lib/js/html5shiv.js"></script>
<![endif]-->
</head>
<body>
<div class="reveal">
<!-- Any section element inside of this container is displayed as a slide -->
<div class="slides">
<section>
<h1>Reveal.js</h1>
<h3>The HTML Presentation Framework</h3>
<p>
<small>Created by <a href="http://hakim.se">Hakim El Hattab</a> and <a href="https://github.com/hakimel/reveal.js/graphs/contributors">contributors</a></small>
</p>
</section>
<section>
<h2>Hello There</h2>
<p>
reveal.js enables you to create beautiful interactive slide decks using HTML. This presentation will show you examples of what it can do.
</p>
</section>
<!-- Example of nested vertical slides -->
<section>
<section>
<h2>Vertical Slides</h2>
<p>Slides can be nested inside of each other.</p>
<p>Use the <em>Space</em> key to navigate through all slides.</p>
<br>
<a href="#" class="navigate-down">
<img width="178" height="238" data-src="https://s3.amazonaws.com/hakim-static/reveal-js/arrow.png" alt="Down arrow">
</a>
</section>
<section>
<h2>Basement Level 1</h2>
<p>Nested slides are useful for adding additional detail underneath a high level horizontal slide.</p>
</section>
<section>
<h2>Basement Level 2</h2>
<p>That's it, time to go back up.</p>
<br>
<a href="#/2">
<img width="178" height="238" data-src="https://s3.amazonaws.com/hakim-static/reveal-js/arrow.png" alt="Up arrow" style="transform: rotate(180deg); -webkit-transform: rotate(180deg);">
</a>
</section>
</section>
<section>
<h2>Slides</h2>
<p>
Not a coder? Not a problem. There's a fully-featured visual editor for authoring these, try it out at <a href="https://slides.com" target="_blank">https://slides.com</a>.
</p>
</section>
<section>
<h2>Point of View</h2>
<p>
Press <strong>ESC</strong> to enter the slide overview.
</p>
<p>
Hold down alt and click on any element to zoom in on it using <a href="http://lab.hakim.se/zoom-js">zoom.js</a>. Alt + click anywhere to zoom back out.
</p>
</section>
<section>
<h2>Touch Optimized</h2>
<p>
Presentations look great on touch devices, like mobile phones and tablets. Simply swipe through your slides.
</p>
</section>
<section data-markdown>
<script type="text/template">
## Markdown support
Write content using inline or external Markdown.
Instructions and more info available in the [readme](https://github.com/hakimel/reveal.js#markdown).
```
<section data-markdown>
## Markdown support
Write content using inline or external Markdown.
Instructions and more info available in the [readme](https://github.com/hakimel/reveal.js#markdown).
</section>
```
</script>
</section>
<section>
<section id="fragments">
<h2>Fragments</h2>
<p>Hit the next arrow...</p>
<p class="fragment">... to step through ...</p>
<p><span class="fragment">... a</span> <span class="fragment">fragmented</span> <span class="fragment">slide.</span></p>
<aside class="notes">
This slide has fragments which are also stepped through in the notes window.
</aside>
</section>
<section>
<h2>Fragment Styles</h2>
<p>There's different types of fragments, like:</p>
<p class="fragment grow">grow</p>
<p class="fragment shrink">shrink</p>
<p class="fragment fade-out">fade-out</p>
<p class="fragment fade-up">fade-up (also down, left and right!)</p>
<p class="fragment current-visible">current-visible</p>
<p>Highlight <span class="fragment highlight-red">red</span> <span class="fragment highlight-blue">blue</span> <span class="fragment highlight-green">green</span></p>
</section>
</section>
<section id="transitions">
<h2>Transition Styles</h2>
<p>
You can select from different transitions, like: <br>
<a href="?transition=none#/transitions">None</a> -
<a href="?transition=fade#/transitions">Fade</a> -
<a href="?transition=slide#/transitions">Slide</a> -
<a href="?transition=convex#/transitions">Convex</a> -
<a href="?transition=concave#/transitions">Concave</a> -
<a href="?transition=zoom#/transitions">Zoom</a>
</p>
</section>
<section id="themes">
<h2>Themes</h2>
<p>
reveal.js comes with a few themes built in: <br>
<!-- Hacks to swap themes after the page has loaded. Not flexible and only intended for the reveal.js demo deck. -->
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/black.css'); return false;">Black (default)</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/white.css'); return false;">White</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/league.css'); return false;">League</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/sky.css'); return false;">Sky</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/beige.css'); return false;">Beige</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/simple.css'); return false;">Simple</a> <br>
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/serif.css'); return false;">Serif</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/blood.css'); return false;">Blood</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/night.css'); return false;">Night</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/moon.css'); return false;">Moon</a> -
<a href="#" onclick="document.getElementById('theme').setAttribute('href','css/theme/solarized.css'); return false;">Solarized</a>
</p>
</section>
<section>
<section data-background="#dddddd">
<h2>Slide Backgrounds</h2>
<p>
Set <code>data-background="#dddddd"</code> on a slide to change the background color. All CSS color formats are supported.
</p>
<a href="#" class="navigate-down">
<img width="178" height="238" data-src="https://s3.amazonaws.com/hakim-static/reveal-js/arrow.png" alt="Down arrow">
</a>
</section>
<section data-background="https://s3.amazonaws.com/hakim-static/reveal-js/image-placeholder.png">
<h2>Image Backgrounds</h2>
<pre><code class="hljs">&lt;section data-background="image.png"&gt;</code></pre>
</section>
<section data-background="https://s3.amazonaws.com/hakim-static/reveal-js/image-placeholder.png" data-background-repeat="repeat" data-background-size="100px">
<h2>Tiled Backgrounds</h2>
<pre><code class="hljs" style="word-wrap: break-word;">&lt;section data-background="image.png" data-background-repeat="repeat" data-background-size="100px"&gt;</code></pre>
</section>
<section data-background-video="https://s3.amazonaws.com/static.slid.es/site/homepage/v1/homepage-video-editor.mp4,https://s3.amazonaws.com/static.slid.es/site/homepage/v1/homepage-video-editor.webm" data-background-color="#000000">
<div style="background-color: rgba(0, 0, 0, 0.9); color: #fff; padding: 20px;">
<h2>Video Backgrounds</h2>
<pre><code class="hljs" style="word-wrap: break-word;">&lt;section data-background-video="video.mp4,video.webm"&gt;</code></pre>
</div>
</section>
<section data-background="http://i.giphy.com/90F8aUepslB84.gif">
<h2>... and GIFs!</h2>
</section>
</section>
<section data-transition="slide" data-background="#4d7e65" data-background-transition="zoom">
<h2>Background Transitions</h2>
<p>
Different background transitions are available via the backgroundTransition option. This one's called "zoom".
</p>
<pre><code class="hljs">Reveal.configure({ backgroundTransition: 'zoom' })</code></pre>
</section>
<section data-transition="slide" data-background="#b5533c" data-background-transition="zoom">
<h2>Background Transitions</h2>
<p>
You can override background transitions per-slide.
</p>
<pre><code class="hljs" style="word-wrap: break-word;">&lt;section data-background-transition="zoom"&gt;</code></pre>
</section>
<section>
<h2>Pretty Code</h2>
<pre><code class="hljs" data-trim contenteditable>
function linkify( selector ) {
if( supports3DTransforms ) {
var nodes = document.querySelectorAll( selector );
for( var i = 0, len = nodes.length; i &lt; len; i++ ) {
var node = nodes[i];
if( !node.className ) {
node.className += ' roll';
}
}
}
}
</code></pre>
<p>Code syntax highlighting courtesy of <a href="http://softwaremaniacs.org/soft/highlight/en/description/">highlight.js</a>.</p>
</section>
<section>
<h2>Marvelous List</h2>
<ul>
<li>No order here</li>
<li>Or here</li>
<li>Or here</li>
<li>Or here</li>
</ul>
</section>
<section>
<h2>Fantastic Ordered List</h2>
<ol>
<li>One is smaller than...</li>
<li>Two is smaller than...</li>
<li>Three!</li>
</ol>
</section>
<section>
<h2>Tabular Tables</h2>
<table>
<thead>
<tr>
<th>Item</th>
<th>Value</th>
<th>Quantity</th>
</tr>
</thead>
<tbody>
<tr>
<td>Apples</td>
<td>$1</td>
<td>7</td>
</tr>
<tr>
<td>Lemonade</td>
<td>$2</td>
<td>18</td>
</tr>
<tr>
<td>Bread</td>
<td>$3</td>
<td>2</td>
</tr>
</tbody>
</table>
</section>
<section>
<h2>Clever Quotes</h2>
<p>
These guys come in two forms, inline: <q cite="http://searchservervirtualization.techtarget.com/definition/Our-Favorite-Technology-Quotations">The nice thing about standards is that there are so many to choose from</q> and block:
</p>
<blockquote cite="http://searchservervirtualization.techtarget.com/definition/Our-Favorite-Technology-Quotations">
&ldquo;For years there has been a theory that millions of monkeys typing at random on millions of typewriters would
reproduce the entire works of Shakespeare. The Internet has proven this theory to be untrue.&rdquo;
</blockquote>
</section>
<section>
<h2>Intergalactic Interconnections</h2>
<p>
You can link between slides internally,
<a href="#/2/3">like this</a>.
</p>
</section>
<section>
<h2>Speaker View</h2>
<p>There's a <a href="https://github.com/hakimel/reveal.js#speaker-notes">speaker view</a>. It includes a timer, preview of the upcoming slide as well as your speaker notes.</p>
<p>Press the <em>S</em> key to try it out.</p>
<aside class="notes">
Oh hey, these are some notes. They'll be hidden in your presentation, but you can see them if you open the speaker notes window (hit 's' on your keyboard).
</aside>
</section>
<section>
<h2>Export to PDF</h2>
<p>Presentations can be <a href="https://github.com/hakimel/reveal.js#pdf-export">exported to PDF</a>, here's an example:</p>
<iframe data-src="https://www.slideshare.net/slideshow/embed_code/42840540" width="445" height="355" frameborder="0" marginwidth="0" marginheight="0" scrolling="no" style="border:3px solid #666; margin-bottom:5px; max-width: 100%;" allowfullscreen> </iframe>
</section>
<section>
<h2>Global State</h2>
<p>
Set <code>data-state="something"</code> on a slide and <code>"something"</code>
will be added as a class to the document element when the slide is open. This lets you
apply broader style changes, like switching the page background.
</p>
</section>
<section data-state="customevent">
<h2>State Events</h2>
<p>
Additionally custom events can be triggered on a per slide basis by binding to the <code>data-state</code> name.
</p>
<pre><code class="javascript" data-trim contenteditable style="font-size: 18px;">
Reveal.addEventListener( 'customevent', function() {
console.log( '"customevent" has fired' );
} );
</code></pre>
</section>
<section>
<h2>Take a Moment</h2>
<p>
Press B or . on your keyboard to pause the presentation. This is helpful when you're on stage and want to take distracting slides off the screen.
</p>
</section>
<section>
<h2>Much more</h2>
<ul>
<li>Right-to-left support</li>
<li><a href="https://github.com/hakimel/reveal.js#api">Extensive JavaScript API</a></li>
<li><a href="https://github.com/hakimel/reveal.js#auto-sliding">Auto-progression</a></li>
<li><a href="https://github.com/hakimel/reveal.js#parallax-background">Parallax backgrounds</a></li>
<li><a href="https://github.com/hakimel/reveal.js#keyboard-bindings">Custom keyboard bindings</a></li>
</ul>
</section>
<section style="text-align: left;">
<h1>THE END</h1>
<p>
- <a href="https://slides.com">Try the online editor</a> <br>
- <a href="https://github.com/hakimel/reveal.js">Source code &amp; documentation</a>
</p>
</section>
</div>
</div>
<script src="lib/js/head.min.js"></script>
<script src="js/reveal.js"></script>
<script>
// More info https://github.com/hakimel/reveal.js#configuration
Reveal.initialize({
controls: true,
progress: true,
history: true,
center: true,
transition: 'slide', // none/fade/slide/convex/concave/zoom
// More info https://github.com/hakimel/reveal.js#dependencies
dependencies: [
{ src: 'lib/js/classList.js', condition: function() { return !document.body.classList; } },
{ src: 'plugin/markdown/marked.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } },
{ src: 'plugin/markdown/markdown.js', condition: function() { return !!document.querySelector( '[data-markdown]' ); } },
{ src: 'plugin/highlight/highlight.js', async: true, callback: function() { hljs.initHighlightingOnLoad(); } },
{ src: 'plugin/search/search.js', async: true },
{ src: 'plugin/zoom-js/zoom.js', async: true },
{ src: 'plugin/notes/notes.js', async: true }
]
});
</script>
</body>
</html>
doc/pub/NeuralNet/html/reveal.js/plugin/multiplex/package.json
+19
View File
@@ -0,0 +1,19 @@
{
"name": "reveal-js-multiplex",
"version": "1.0.0",
"description": "reveal.js multiplex server",
"homepage": "http://revealjs.com",
"scripts": {
"start": "node index.js"
},
"engines": {
"node": "~4.1.1"
},
"dependencies": {
"express": "~4.13.3",
"grunt-cli": "~0.1.13",
"mustache": "~2.2.1",
"socket.io": "~1.3.7"
},
"license": "MIT"
}
doc/pub/NeuralNet/html/reveal.js/test/simple.md
+12
View File
@@ -0,0 +1,12 @@
## Slide 1.1
```js
var a = 1;
```
## Slide 1.2
## Slide 2
doc/pub/NeuralNet/html/reveal.js/test/test-markdown-external.html
+36
View File
@@ -0,0 +1,36 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>reveal.js - Test Markdown</title>
<link rel="stylesheet" href="../css/reveal.css">
<link rel="stylesheet" href="qunit-1.12.0.css">
</head>
<body style="overflow: auto;">
<div id="qunit"></div>
<div id="qunit-fixture"></div>
<div class="reveal" style="display: none;">
<div class="slides">
<section data-markdown="simple.md" data-separator="^\n\n\n" data-separator-vertical="^\n\n"></section>
</div>
</div>
<script src="../lib/js/head.min.js"></script>
<script src="../js/reveal.js"></script>
<script src="../plugin/highlight/highlight.js"></script>
<script src="../plugin/markdown/marked.js"></script>
<script src="../plugin/markdown/markdown.js"></script>
<script src="qunit-1.12.0.js"></script>
<script src="test-markdown-external.js"></script>
</body>
</html>
doc/pub/NeuralNet/html/reveal.js/test/test-markdown-external.js
+24
View File
@@ -0,0 +1,24 @@
Reveal.addEventListener( 'ready', function() {
QUnit.module( 'Markdown' );
test( 'Vertical separator', function() {
strictEqual( document.querySelectorAll( '.reveal .slides>section>section' ).length, 2, 'found two slides' );
});
test( 'Horizontal separator', function() {
strictEqual( document.querySelectorAll( '.reveal .slides>section' ).length, 2, 'found two slides' );
});
test( 'Language highlighter', function() {
strictEqual( document.querySelectorAll( '.hljs-keyword' ).length, 1, 'got rendered highlight tag.' );
strictEqual( document.querySelector( '.hljs-keyword' ).innerHTML, 'var', 'the same keyword: var.' );
});
} );
Reveal.initialize();
doc/pub/NeuralNet/html/reveal.js/test/test-markdown-options.html
+41
View File
@@ -0,0 +1,41 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>reveal.js - Test Markdown Options</title>
<link rel="stylesheet" href="../css/reveal.css">
<link rel="stylesheet" href="qunit-1.12.0.css">
</head>
<body style="overflow: auto;">
<div id="qunit"></div>
<div id="qunit-fixture"></div>
<div class="reveal" style="display: none;">
<div class="slides">
<section data-markdown>
<script type="text/template">
## Testing Markdown Options
This "slide" should contain 'smart' quotes.
</script>
</section>
</div>
</div>
<script src="../lib/js/head.min.js"></script>
<script src="../js/reveal.js"></script>
<script src="qunit-1.12.0.js"></script>
<script src="test-markdown-options.js"></script>
</body>
</html>
doc/pub/NeuralNet/html/reveal.js/test/test-markdown-options.js
+26
View File
@@ -0,0 +1,26 @@
Reveal.addEventListener( 'ready', function() {
QUnit.module( 'Markdown' );
test( 'Options are set', function() {
strictEqual( marked.defaults.smartypants, true );
});
test( 'Smart quotes are activated', function() {
var text = document.querySelector( '.reveal .slides>section>p' ).textContent;
strictEqual( /['"]/.test( text ), false );
strictEqual( /[“”‘’]/.test( text ), true );
});
} );
Reveal.initialize({
dependencies: [
{ src: '../plugin/markdown/marked.js' },
{ src: '../plugin/markdown/markdown.js' },
],
markdown: {
smartypants: true
}
});
doc/pub/NeuralNet/ipynb/.ipynb_checkpoints/NeuralNet-checkpoint.ipynb
+1163
View File
File diff suppressed because one or more lines are too long
doc/pub/NeuralNet/ipynb/.ipynb_checkpoints/Untitled-checkpoint.ipynb
+6
View File
@@ -0,0 +1,6 @@
{
"cells": [],
"metadata": {},
"nbformat": 4,
"nbformat_minor": 2
}

Some files were not shown because too many files have changed in this diff Show More