rewriting reg analysis

doc/pub/Regression/ipynb/Regression.ipynb

@@ -1649,9 +1649,67 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 11,
+   "execution_count": 1,
    "metadata": {},
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "X_train:  (37, 1)\n",
+      "y_train:  (37,)\n",
+      "X_test:  (13, 1)\n",
+      "y_test:  (13,)\n",
+      "------------------------------------\n",
+      "Ordinary Least Squares\n",
+      "Prediction Shape:  (13,)\n",
+      "Coefficients: \n",
+      " [0.54090544]\n",
+      "Mean squared error: 4.92\n",
+      "Variance score: 0.11\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/usr/local/lib/python3.7/site-packages/sklearn/linear_model/base.py:509: RuntimeWarning: internal gelsd driver lwork query error, required iwork dimension not returned. This is likely the result of LAPACK bug 0038, fixed in LAPACK 3.2.2 (released July 21, 2010). Falling back to 'gelss' driver.\n",
+      "  linalg.lstsq(X, y)\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "<Figure size 640x480 with 1 Axes>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "------------------------------------\n",
+      "Ridge Regression\n",
+      "Ridge Coefficient:  [0.54089262]\n",
+      "Ridge Intercept:  4.6076494432587065\n",
+      "------------------------------------\n",
+      "Lasso\n",
+      "Lasso Coefficient:  [0.54002825]\n",
+      "Lasso Intercept:  4.623208095009552\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "<Figure size 640x480 with 1 Axes>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
    "source": [
     "import matplotlib.pyplot as plt\n",
     "import numpy as np\n",

doc/src/Regression/Regression.do.txt

@@ -938,18 +938,22 @@ plt.show()
 !split
 ===== The singular value decompostion  =====
 !bblock
-How can we use the singular value decomposition to find the parameters $\beta_j$? More details will come. We first note that a general $m\times n$ matrix $\hat{A}$ can be written in terms of a diagonal matrix $\hat{\Sigma}$ of dimensionality $n\times n$ and two orthognal matrices $\hat{U}$ and $\hat{V}$, where the first has dimensionality $m \times n$ and the last dimensionality $n\times n$. We have then
+
+
+A general
+$m\times n$ matrix $\hat{A}$ can be written in terms of a diagonal
+matrix $\hat{D}$ of dimensionality $n\times n$ and two orthognal
+matrices $\hat{U}$ and $\hat{V}$, where the first has dimensionality
+$m \times m$ and the last dimensionality $n\times n$. 
+We have then 
 !bt
-\[
-\hat{A} = \hat{U}\hat{\Sigma}\hat{V}
-\]
+\[ 
+\hat{A} = \hat{U}\hat{D}\hat{V}^T 
+\] 
 !et 
 !eblock
 
 
-Add codes and discuss this in connection with lasso and ridge, show example where the standard inversion of a matrix fails and where SVD comes to rescue. Include first a discussion of the general SVD and then discuss the thin SVD.
-
-
 !split
 ===== Lasso and Ridge regression =====
 
@@ -957,7 +961,7 @@ Discuss the mathematics here
 
 
 !split
-===== Ridge and Lasso Regression =====
+===== Code examples for Ridge and Lasso Regression =====
 
 !bc pycod
 import matplotlib.pyplot as plt