diff --git a/doc/pub/How2ReadData/html/How2ReadData-bs.html b/doc/pub/How2ReadData/html/How2ReadData-bs.html index a3bdb97d5..77c33d5d6 100644 --- a/doc/pub/How2ReadData/html/How2ReadData-bs.html +++ b/doc/pub/How2ReadData/html/How2ReadData-bs.html @@ -57,7 +57,8 @@ Automatically generated HTML file from DocOnce source ('Particle in one dimension and velocity distribution', 3, None, - '___sec11')]} + '___sec11'), + ('Random walk model', 3, None, '___sec12')]} end of tocinfo --> @@ -107,6 +108,7 @@ MathJax.Hub.Config({
  •    Predator-Prey model from ecology
  •    Simulating financial transactions
  •    Particle in one dimension and velocity distribution
    • +
  •    Random walk model
    • @@ -143,11 +145,8 @@ MathJax.Hub.Config({

    May 29, 2018


    - - -

    Introduction

    @@ -188,9 +187,6 @@ introduce will serve as inputs to many of our discussions later, as well as allowing you to set up models and produce your own data and get started with programming. -

    - -

    Software and needed installations

    @@ -233,9 +229,6 @@ you can use pip as well and simply install Python as etc etc. -

    - -

    Python installers

    @@ -263,9 +256,6 @@ distribution for scientific and analytic computing distribution and analysis environment, available for free and under a commercial license. -

    - -

    Installing R, C++, cython or Julia

    @@ -284,9 +274,6 @@ texts. To install R with Jupyter notebook follow the link here -

    - -

    Installing R, C++, cython, Numba etc

    @@ -321,9 +308,6 @@ Finally, if you wish to use the light mark-up language doconce you can convert a standard ascii text file into various HTML formats, ipython notebooks, latex files, pdf files etc with minimal edits. -

    - -

    Simple linear regression model using scikit-learn

    @@ -610,7 +594,6 @@ plt.show()

    Similarly, using R, we can perform similar studies. The following R code illustrates this. -

    Non-Linear Least squares in R

    @@ -662,8 +645,6 @@ data = {'Na data_pandas = pd.DataFrame(data) display(data_pandas)
    -

    -

    Examples

    @@ -1332,6 +1313,101 @@ plt.axis([-5.grid(True) plt.show() + +

    Random walk model

    + +

    + + +

    import numpy as np
+import matplotlib.pyplot as plt
+from sklearn.preprocessing import PolynomialFeatures
+from sklearn.linear_model import LinearRegression
+
+steps=250
+
+distance=0
+x=0
+distance_list=[]
+steps_list=[]
+while x<steps:
+    distance+=np.random.randint(-1,2)
+    distance_list.append(distance)
+    x+=1
+    steps_list.append(x)
+plt.plot(steps_list,distance_list, color='green', label="Random Walk Data")
+
+steps_list=np.asarray(steps_list)
+distance_list=np.asarray(distance_list)
+
+X=steps_list[:,np.newaxis]
+
+#Polynomial fits
+
+#Degree 2
+poly_features=PolynomialFeatures(degree=2, include_bias=False)
+X_poly=poly_features.fit_transform(X)
+
+lin_reg=LinearRegression()
+poly_fit=lin_reg.fit(X_poly,distance_list)
+b=lin_reg.coef_
+c=lin_reg.intercept_
+print ("2nd degree coefficients:")
+print ("zero power: ",c)
+print ("first power: ", b[0])
+print ("second power: ",b[1])
+
+z = np.arange(0, steps, .01)
+z_mod=b[1]*z**2+b[0]*z+c
+
+fit_mod=b[1]*X**2+b[0]*X+c
+plt.plot(z, z_mod, color='r', label="2nd Degree Fit")
+plt.title("Polynomial Regression")
+
+plt.xlabel("Steps")
+plt.ylabel("Distance")
+
+#Degree 10
+poly_features10=PolynomialFeatures(degree=10, include_bias=False)
+X_poly10=poly_features10.fit_transform(X)
+
+poly_fit10=lin_reg.fit(X_poly10,distance_list)
+
+y_plot=poly_fit10.predict(X_poly10)
+plt.plot(X, y_plot, color='black', label="10th Degree Fit")
+
+plt.legend()
+plt.show()
+
+
+#Decision Tree Regression
+from sklearn.tree import DecisionTreeRegressor
+regr_1=DecisionTreeRegressor(max_depth=2)
+regr_2=DecisionTreeRegressor(max_depth=5)
+regr_3=DecisionTreeRegressor(max_depth=7)
+regr_1.fit(X, distance_list)
+regr_2.fit(X, distance_list)
+regr_3.fit(X, distance_list)
+
+X_test = np.arange(0.0, steps, 0.01)[:, np.newaxis]
+y_1 = regr_1.predict(X_test)
+y_2 = regr_2.predict(X_test)
+y_3=regr_3.predict(X_test)
+
+# Plot the results
+plt.figure()
+plt.scatter(X, distance_list, s=2.5, c="black", label="data")
+plt.plot(X_test, y_1, color="red",
+         label="max_depth=2", linewidth=2)
+plt.plot(X_test, y_2, color="green", label="max_depth=5", linewidth=2)
+plt.plot(X_test, y_3, color="m", label="max_depth=7", linewidth=2)
+
+plt.xlabel("Data")
+plt.ylabel("Darget")
+plt.title("Decision Tree Regression")
+plt.legend()
+plt.show()
+

    diff --git a/doc/pub/How2ReadData/html/How2ReadData-reveal.html b/doc/pub/How2ReadData/html/How2ReadData-reveal.html index d0879d153..34e634731 100644 --- a/doc/pub/How2ReadData/html/How2ReadData-reveal.html +++ b/doc/pub/How2ReadData/html/How2ReadData-reveal.html @@ -150,15 +150,7 @@ MathJax.Hub.Config({

     

    May 29, 2018


    -

    -

    - © 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license -
    - - - -

    Introduction

    @@ -198,10 +190,7 @@ tensorflow (see below for links etc). Moreover, the examples we introduce will serve as inputs to many of our discussions later, as well as allowing you to set up models and produce your own data and get started with programming. -

    - -

    Software and needed installations

    @@ -246,10 +235,7 @@ you can use pip as well and simply install Python as

    etc etc. -

    - -

    Python installers

    @@ -278,10 +264,7 @@ is a Python distribution for scientific and analytic computing distribution and analysis environment, available for free and under a commercial license. -

    - -

    Installing R, C++, cython or Julia

    @@ -299,10 +282,7 @@ texts.

    To install R with Jupyter notebook follow the link here -

    - -

    Installing R, C++, cython, Numba etc

    @@ -336,10 +316,7 @@ And to add more versatility, the Python package doconce you can convert a standard ascii text file into various HTML formats, ipython notebooks, latex files, pdf files etc with minimal edits. -

    - -

    Simple linear regression model using scikit-learn

    @@ -644,10 +621,7 @@ plt.show()

    Similarly, using R, we can perform similar studies. The following R code illustrates this. -

    - -

    Non-Linear Least squares in R

    @@ -695,10 +669,7 @@ data = {'Name': [Examples

    @@ -1396,6 +1367,107 @@ plt.axis([-5, 5 plt.grid(True) plt.show()

    + +

    Random walk model

    + +

    + + +

    import numpy as np
+import matplotlib.pyplot as plt
+from sklearn.preprocessing import PolynomialFeatures
+from sklearn.linear_model import LinearRegression
+
+steps=250
+
+distance=0
+x=0
+distance_list=[]
+steps_list=[]
+while x<steps:
+    distance+=np.random.randint(-1,2)
+    distance_list.append(distance)
+    x+=1
+    steps_list.append(x)
+plt.plot(steps_list,distance_list, color='green', label="Random Walk Data")
+
+steps_list=np.asarray(steps_list)
+distance_list=np.asarray(distance_list)
+
+X=steps_list[:,np.newaxis]
+
+#Polynomial fits
+
+#Degree 2
+poly_features=PolynomialFeatures(degree=2, include_bias=False)
+X_poly=poly_features.fit_transform(X)
+
+lin_reg=LinearRegression()
+poly_fit=lin_reg.fit(X_poly,distance_list)
+b=lin_reg.coef_
+c=lin_reg.intercept_
+print ("2nd degree coefficients:")
+print ("zero power: ",c)
+print ("first power: ", b[0])
+print ("second power: ",b[1])
+
+z = np.arange(0, steps, .01)
+z_mod=b[1]*z**2+b[0]*z+c
+
+fit_mod=b[1]*X**2+b[0]*X+c
+plt.plot(z, z_mod, color='r', label="2nd Degree Fit")
+plt.title("Polynomial Regression")
+
+plt.xlabel("Steps")
+plt.ylabel("Distance")
+
+#Degree 10
+poly_features10=PolynomialFeatures(degree=10, include_bias=False)
+X_poly10=poly_features10.fit_transform(X)
+
+poly_fit10=lin_reg.fit(X_poly10,distance_list)
+
+y_plot=poly_fit10.predict(X_poly10)
+plt.plot(X, y_plot, color='black', label="10th Degree Fit")
+
+plt.legend()
+plt.show()
+
+
+#Decision Tree Regression
+from sklearn.tree import DecisionTreeRegressor
+regr_1=DecisionTreeRegressor(max_depth=2)
+regr_2=DecisionTreeRegressor(max_depth=5)
+regr_3=DecisionTreeRegressor(max_depth=7)
+regr_1.fit(X, distance_list)
+regr_2.fit(X, distance_list)
+regr_3.fit(X, distance_list)
+
+X_test = np.arange(0.0, steps, 0.01)[:, np.newaxis]
+y_1 = regr_1.predict(X_test)
+y_2 = regr_2.predict(X_test)
+y_3=regr_3.predict(X_test)
+
+# Plot the results
+plt.figure()
+plt.scatter(X, distance_list, s=2.5, c="black", label="data")
+plt.plot(X_test, y_1, color="red",
+         label="max_depth=2", linewidth=2)
+plt.plot(X_test, y_2, color="green", label="max_depth=5", linewidth=2)
+plt.plot(X_test, y_3, color="m", label="max_depth=7", linewidth=2)
+
+plt.xlabel("Data")
+plt.ylabel("Darget")
+plt.title("Decision Tree Regression")
+plt.legend()
+plt.show()
+
    +

    + + +

    + © 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license +
    diff --git a/doc/pub/How2ReadData/html/How2ReadData-solarized.html b/doc/pub/How2ReadData/html/How2ReadData-solarized.html index dd677c506..a84d765ae 100644 --- a/doc/pub/How2ReadData/html/How2ReadData-solarized.html +++ b/doc/pub/How2ReadData/html/How2ReadData-solarized.html @@ -77,7 +77,8 @@ div { text-align: justify; text-justify: inter-word; } ('Particle in one dimension and velocity distribution', 3, None, - '___sec11')]} + '___sec11'), + ('Random walk model', 3, None, '___sec12')]} end of tocinfo --> @@ -121,8 +122,6 @@ MathJax.Hub.Config({

    May 29, 2018


    -

    -









    Introduction

    @@ -164,9 +163,6 @@ introduce will serve as inputs to many of our discussions later, as well as allowing you to set up models and produce your own data and get started with programming. -

    -









    -

    Software and needed installations

    @@ -209,9 +205,6 @@ you can use pip as well and simply install Python as etc etc. -

    -









    -

    Python installers

    @@ -239,9 +232,6 @@ distribution for scientific and analytic computing distribution and analysis environment, available for free and under a commercial license. -

    -









    -

    Installing R, C++, cython or Julia

    @@ -260,9 +250,6 @@ texts. To install R with Jupyter notebook follow the link here -

    -









    -

    Installing R, C++, cython, Numba etc

    @@ -297,9 +284,6 @@ Finally, if you wish to use the light mark-up language doconce you can convert a standard ascii text file into various HTML formats, ipython notebooks, latex files, pdf files etc with minimal edits. -

    -









    -

    Simple linear regression model using scikit-learn

    @@ -586,7 +570,6 @@ plt.show()

    Similarly, using R, we can perform similar studies. The following R code illustrates this. -









    Non-Linear Least squares in R

    @@ -637,8 +620,6 @@ data = {'Name': [Examples @@ -1300,6 +1281,101 @@ plt.axis([-5, 5 plt.grid(True) plt.show()
    + +

    Random walk model

    + +

    + + +

    import numpy as np
+import matplotlib.pyplot as plt
+from sklearn.preprocessing import PolynomialFeatures
+from sklearn.linear_model import LinearRegression
+
+steps=250
+
+distance=0
+x=0
+distance_list=[]
+steps_list=[]
+while x<steps:
+    distance+=np.random.randint(-1,2)
+    distance_list.append(distance)
+    x+=1
+    steps_list.append(x)
+plt.plot(steps_list,distance_list, color='green', label="Random Walk Data")
+
+steps_list=np.asarray(steps_list)
+distance_list=np.asarray(distance_list)
+
+X=steps_list[:,np.newaxis]
+
+#Polynomial fits
+
+#Degree 2
+poly_features=PolynomialFeatures(degree=2, include_bias=False)
+X_poly=poly_features.fit_transform(X)
+
+lin_reg=LinearRegression()
+poly_fit=lin_reg.fit(X_poly,distance_list)
+b=lin_reg.coef_
+c=lin_reg.intercept_
+print ("2nd degree coefficients:")
+print ("zero power: ",c)
+print ("first power: ", b[0])
+print ("second power: ",b[1])
+
+z = np.arange(0, steps, .01)
+z_mod=b[1]*z**2+b[0]*z+c
+
+fit_mod=b[1]*X**2+b[0]*X+c
+plt.plot(z, z_mod, color='r', label="2nd Degree Fit")
+plt.title("Polynomial Regression")
+
+plt.xlabel("Steps")
+plt.ylabel("Distance")
+
+#Degree 10
+poly_features10=PolynomialFeatures(degree=10, include_bias=False)
+X_poly10=poly_features10.fit_transform(X)
+
+poly_fit10=lin_reg.fit(X_poly10,distance_list)
+
+y_plot=poly_fit10.predict(X_poly10)
+plt.plot(X, y_plot, color='black', label="10th Degree Fit")
+
+plt.legend()
+plt.show()
+
+
+#Decision Tree Regression
+from sklearn.tree import DecisionTreeRegressor
+regr_1=DecisionTreeRegressor(max_depth=2)
+regr_2=DecisionTreeRegressor(max_depth=5)
+regr_3=DecisionTreeRegressor(max_depth=7)
+regr_1.fit(X, distance_list)
+regr_2.fit(X, distance_list)
+regr_3.fit(X, distance_list)
+
+X_test = np.arange(0.0, steps, 0.01)[:, np.newaxis]
+y_1 = regr_1.predict(X_test)
+y_2 = regr_2.predict(X_test)
+y_3=regr_3.predict(X_test)
+
+# Plot the results
+plt.figure()
+plt.scatter(X, distance_list, s=2.5, c="black", label="data")
+plt.plot(X_test, y_1, color="red",
+         label="max_depth=2", linewidth=2)
+plt.plot(X_test, y_2, color="green", label="max_depth=5", linewidth=2)
+plt.plot(X_test, y_3, color="m", label="max_depth=7", linewidth=2)
+
+plt.xlabel("Data")
+plt.ylabel("Darget")
+plt.title("Decision Tree Regression")
+plt.legend()
+plt.show()
+

    diff --git a/doc/pub/How2ReadData/html/How2ReadData.html b/doc/pub/How2ReadData/html/How2ReadData.html index c401e5a8f..ee631f1c0 100644 --- a/doc/pub/How2ReadData/html/How2ReadData.html +++ b/doc/pub/How2ReadData/html/How2ReadData.html @@ -82,7 +82,8 @@ div { text-align: justify; text-justify: inter-word; } ('Particle in one dimension and velocity distribution', 3, None, - '___sec11')]} + '___sec11'), + ('Random walk model', 3, None, '___sec12')]} end of tocinfo --> @@ -126,8 +127,6 @@ MathJax.Hub.Config({

    May 29, 2018


    -

    -









    Introduction

    @@ -169,9 +168,6 @@ introduce will serve as inputs to many of our discussions later, as well as allowing you to set up models and produce your own data and get started with programming. -

    -









    -

    Software and needed installations

    @@ -214,9 +210,6 @@ you can use pip as well and simply install Python as etc etc. -

    -









    -

    Python installers

    @@ -244,9 +237,6 @@ distribution for scientific and analytic computing distribution and analysis environment, available for free and under a commercial license. -

    -









    -

    Installing R, C++, cython or Julia

    @@ -265,9 +255,6 @@ texts. To install R with Jupyter notebook follow the link here -

    -









    -

    Installing R, C++, cython, Numba etc

    @@ -302,9 +289,6 @@ Finally, if you wish to use the light mark-up language doconce you can convert a standard ascii text file into various HTML formats, ipython notebooks, latex files, pdf files etc with minimal edits. -

    -









    -

    Simple linear regression model using scikit-learn

    @@ -591,7 +575,6 @@ plt.show()

    Similarly, using R, we can perform similar studies. The following R code illustrates this. -









    Non-Linear Least squares in R

    @@ -642,8 +625,6 @@ data = {'Na data_pandas = pd.DataFrame(data) display(data_pandas)
    -

    -









    Examples

    @@ -1305,6 +1286,101 @@ plt.axis([-5.grid(True) plt.show() + +

    Random walk model

    + +

    + + +

    import numpy as np
+import matplotlib.pyplot as plt
+from sklearn.preprocessing import PolynomialFeatures
+from sklearn.linear_model import LinearRegression
+
+steps=250
+
+distance=0
+x=0
+distance_list=[]
+steps_list=[]
+while x<steps:
+    distance+=np.random.randint(-1,2)
+    distance_list.append(distance)
+    x+=1
+    steps_list.append(x)
+plt.plot(steps_list,distance_list, color='green', label="Random Walk Data")
+
+steps_list=np.asarray(steps_list)
+distance_list=np.asarray(distance_list)
+
+X=steps_list[:,np.newaxis]
+
+#Polynomial fits
+
+#Degree 2
+poly_features=PolynomialFeatures(degree=2, include_bias=False)
+X_poly=poly_features.fit_transform(X)
+
+lin_reg=LinearRegression()
+poly_fit=lin_reg.fit(X_poly,distance_list)
+b=lin_reg.coef_
+c=lin_reg.intercept_
+print ("2nd degree coefficients:")
+print ("zero power: ",c)
+print ("first power: ", b[0])
+print ("second power: ",b[1])
+
+z = np.arange(0, steps, .01)
+z_mod=b[1]*z**2+b[0]*z+c
+
+fit_mod=b[1]*X**2+b[0]*X+c
+plt.plot(z, z_mod, color='r', label="2nd Degree Fit")
+plt.title("Polynomial Regression")
+
+plt.xlabel("Steps")
+plt.ylabel("Distance")
+
+#Degree 10
+poly_features10=PolynomialFeatures(degree=10, include_bias=False)
+X_poly10=poly_features10.fit_transform(X)
+
+poly_fit10=lin_reg.fit(X_poly10,distance_list)
+
+y_plot=poly_fit10.predict(X_poly10)
+plt.plot(X, y_plot, color='black', label="10th Degree Fit")
+
+plt.legend()
+plt.show()
+
+
+#Decision Tree Regression
+from sklearn.tree import DecisionTreeRegressor
+regr_1=DecisionTreeRegressor(max_depth=2)
+regr_2=DecisionTreeRegressor(max_depth=5)
+regr_3=DecisionTreeRegressor(max_depth=7)
+regr_1.fit(X, distance_list)
+regr_2.fit(X, distance_list)
+regr_3.fit(X, distance_list)
+
+X_test = np.arange(0.0, steps, 0.01)[:, np.newaxis]
+y_1 = regr_1.predict(X_test)
+y_2 = regr_2.predict(X_test)
+y_3=regr_3.predict(X_test)
+
+# Plot the results
+plt.figure()
+plt.scatter(X, distance_list, s=2.5, c="black", label="data")
+plt.plot(X_test, y_1, color="red",
+         label="max_depth=2", linewidth=2)
+plt.plot(X_test, y_2, color="green", label="max_depth=5", linewidth=2)
+plt.plot(X_test, y_3, color="m", label="max_depth=7", linewidth=2)
+
+plt.xlabel("Data")
+plt.ylabel("Darget")
+plt.title("Decision Tree Regression")
+plt.legend()
+plt.show()
+

    diff --git a/doc/pub/How2ReadData/ipynb/How2ReadData.ipynb b/doc/pub/How2ReadData/ipynb/How2ReadData.ipynb index 106fc8862..95954d7ff 100644 --- a/doc/pub/How2ReadData/ipynb/How2ReadData.ipynb +++ b/doc/pub/How2ReadData/ipynb/How2ReadData.ipynb @@ -17,6 +17,7 @@ "\n", "\n", "\n", + "\n", "## Introduction\n", "\n", "Our emphasis throughout this series of lectures \n", @@ -57,6 +58,7 @@ "\n", "\n", "\n", + "\n", "## Software and needed installations\n", "\n", "We will make extensive use of Python as programming language and its\n", @@ -91,6 +93,7 @@ "\n", "etc etc. \n", "\n", + "\n", "## Python installers\n", "\n", "If you don't want to perform these operations separately and venture\n", @@ -114,6 +117,7 @@ "license.\n", "\n", "\n", + "\n", "## Installing R, C++, cython or Julia\n", "\n", "You will also find it convenient to utilize R. Although we will mainly\n", @@ -132,6 +136,7 @@ "\n", "\n", "\n", + "\n", "## Installing R, C++, cython, Numba etc\n", "\n", "\n", @@ -170,6 +175,7 @@ "[doconce](https://github.com/hplgit/doconce) you can convert a standard ascii text file into various HTML \n", "formats, ipython notebooks, latex files, pdf files etc with minimal edits.\n", "\n", + "\n", "## Simple linear regression model using **scikit-learn**\n", "\n", "We start with perhaps our simplest possible example, using **scikit-learn** to perform linear regression analysis on a data set produced by us. \n", @@ -582,6 +588,8 @@ "metadata": {}, "source": [ "Similarly, using **R**, we can perform similar studies. The following **R** code illustrates this.\n", + "\n", + "\n", "## Non-Linear Least squares in R" ] }, @@ -1573,6 +1581,111 @@ "plt.grid(True)\n", "plt.show()" ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Random walk model" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "import numpy as np\n", + "import matplotlib.pyplot as plt\n", + "from sklearn.preprocessing import PolynomialFeatures\n", + "from sklearn.linear_model import LinearRegression\n", + "\n", + "steps=250\n", + "\n", + "distance=0\n", + "x=0\n", + "distance_list=[]\n", + "steps_list=[]\n", + "while x