+The cases we have studied till were all characterized by two classes +with a close to linear separability. The classifiers we have described +so far find linear boundaries in our input feature space. It is +possible to make our procedure more flexible by exploring the feature +space using other basis expansions such higher-order polynomials, +wavelets, splines etc. + +
+If our feature space is not easy to separate, as shown in the figure +here, we can achieve a better separation by introducing more complex +basis functions. The ideal would be, as shown in the next figure, to, via a specific transformation to +obtain a separation between the classes which is almost linear. + +
+The change of basis, from \( x\rightarrow z=\phi(x) \) leads to the same type of equations to be solved, except that +we need to introduce for example a polynomial transformation to a two-dimensional training set. + +
diff --git a/doc/pub/svm/html/._svm-bs018.html b/doc/pub/svm/html/._svm-bs018.html new file mode 100644 index 000000000..0a17b8423 --- /dev/null +++ b/doc/pub/svm/html/._svm-bs018.html @@ -0,0 +1,207 @@ + + + + + + + +
+ + + + +
+Suppose we define a polynomial transformation of degree two (we continue to live in a plane with \( x_1 \) and \( x_2 \) as variables) +$$ +z = \phi(x) =\left(1, x_1, x_2, x_1^2, x_2^2, x_1x_2). +$$ + +
+With our new basis, the equations we solved earlier are basically the same, that is we have now (without the slack option for simplicity) +$$ +{\cal L}=\sum_i\lambda_i-\frac{1}{2}\sum_{ij}^n\lambda_i\lambda_jy_iy_j\boldsymbol{z}_i^T\boldsymbol{Z}_j, +$$ + +subject to the constraints \( \lambda_i\geq 0 \), \( \sum_i\lambda_iy_i=0 \), and for the support vectors +$$ +y_i(\boldsymbol{w}^T\boldsymbol{z}_i+b)= 1 \hspace{0.1cm}\forall i, +$$ + +from which we also find \( b \). + +
+
+ +
+ + +
+ + + + +
+If we use Python as programming language and wish to venture beyond +scikit-learn, tensorflow and similar software which makes our +lives so much easier, we need to dive into the wonderful world of +quadratic programming. We can, if we wish, solve the minimization +problem using say standard gradient methods or conjugate gradient +methods. However, these methods tend to exhibit a rather slow +converge. So, welcome to the promised land of quadratic programming. + +
+The functions we need are contained in the quadratic programming package CVXOPT and we need to import it +
+ + +
import numpy
+import cvxopt
++Let us first set up the standard form the of quadratic programming (QP) equations by defining the problem as +$$ +\mathrm{min} +$$ + +
+subject to Gx u. Note that x itself is not provided to the solver, since it is an internal +variable being optimized over. In particular, this means that the solver has no explicit knowledge +of x itself; everything is implicity defined by the supplied parameters. It is essential +that the same variable order is maintained for the relevant parameters (e.g., qi +Non-convexity implies the existence of local optima, making it difficult to find global optima. + +
+collapsed all inequality constraints into a single G matrix of the standard form. +Since there are no equality constraints, we do not need to provide the empty A, b. Note +that even though y +2 did not appear in the original objective, we had to include it with zero +coefficients in P because the solver parameters must be defined using the full set of variables. +Even if certain variables only appear in constraints, they will still need to be expressed with +zero coefficients in the objective parameters, and vice versa. +Let us first define the above parameters in Python. CVXOPT supplies its own matrix +object; all arguments given to its solvers must be in this matrix type. There are two ways +to do this. The first is to define the matrix directly with (potentially nested) lists: +from cvxopt import matrix +
+ + +
P = matrix([[1.0,0.0],[0.0,0.0]])
+q = matrix([3.0,4.0])
+G = matrix([[-1.0,0.0,-1.0,2.0,3.0],[0.0,-1.0,-3.0,5.0,4.0]])
+h = matrix([0.0,0.0,-15.0,100.0,80.0])
++ +
+ +
+ + ++The cases we have studied till were all characterized by two classes +with a close to linear separability. The classifiers we have described +so far find linear boundaries in our input feature space. It is +possible to make our procedure more flexible by exploring the feature +space using other basis expansions such higher-order polynomials, +wavelets, splines etc. + +
+If our feature space is not easy to separate, as shown in the figure +here, we can achieve a better separation by introducing more complex +basis functions. The ideal would be, as shown in the next figure, to, via a specific transformation to +obtain a separation between the classes which is almost linear. + +
+The change of basis, from \( x\rightarrow z=\phi(x) \) leads to the same type of equations to be solved, except that +we need to introduce for example a polynomial transformation to a two-dimensional training set. +
+Suppose we define a polynomial transformation of degree two (we continue to live in a plane with \( x_1 \) and \( x_2 \) as variables) +
+$$
+z = \phi(x) =\left(1, x_1, x_2, x_1^2, x_2^2, x_1x_2).
+$$
+
+
+
+With our new basis, the equations we solved earlier are basically the same, that is we have now (without the slack option for simplicity) +
+$$
+{\cal L}=\sum_i\lambda_i-\frac{1}{2}\sum_{ij}^n\lambda_i\lambda_jy_iy_j\boldsymbol{z}_i^T\boldsymbol{Z}_j,
+$$
+
+
+subject to the constraints \( \lambda_i\geq 0 \), \( \sum_i\lambda_iy_i=0 \), and for the support vectors
+
+$$
+y_i(\boldsymbol{w}^T\boldsymbol{z}_i+b)= 1 \hspace{0.1cm}\forall i,
+$$
+
+
+from which we also find \( b \).
+
+If we use Python as programming language and wish to venture beyond +scikit-learn, tensorflow and similar software which makes our +lives so much easier, we need to dive into the wonderful world of +quadratic programming. We can, if we wish, solve the minimization +problem using say standard gradient methods or conjugate gradient +methods. However, these methods tend to exhibit a rather slow +converge. So, welcome to the promised land of quadratic programming. + +
+The functions we need are contained in the quadratic programming package CVXOPT and we need to import it +
+ + +
import numpy
+import cvxopt
++Let us first set up the standard form the of quadratic programming (QP) equations by defining the problem as +
+$$
+\mathrm{min}
+$$
+
+
+
+subject to Gx u. Note that x itself is not provided to the solver, since it is an internal +variable being optimized over. In particular, this means that the solver has no explicit knowledge +of x itself; everything is implicity defined by the supplied parameters. It is essential +that the same variable order is maintained for the relevant parameters (e.g., qi +Non-convexity implies the existence of local optima, making it difficult to find global optima. + +
+collapsed all inequality constraints into a single G matrix of the standard form. +Since there are no equality constraints, we do not need to provide the empty A, b. Note +that even though y +2 did not appear in the original objective, we had to include it with zero +coefficients in P because the solver parameters must be defined using the full set of variables. +Even if certain variables only appear in constraints, they will still need to be expressed with +zero coefficients in the objective parameters, and vice versa. +Let us first define the above parameters in Python. CVXOPT supplies its own matrix +object; all arguments given to its solvers must be in this matrix type. There are two ways +to do this. The first is to define the matrix directly with (potentially nested) lists: +from cvxopt import matrix +
+ + +
P = matrix([[1.0,0.0],[0.0,0.0]])
+q = matrix([3.0,4.0])
+G = matrix([[-1.0,0.0,-1.0,2.0,3.0],[0.0,-1.0,-3.0,5.0,4.0]])
+h = matrix([0.0,0.0,-15.0,100.0,80.0])
++The cases we have studied till were all characterized by two classes +with a close to linear separability. The classifiers we have described +so far find linear boundaries in our input feature space. It is +possible to make our procedure more flexible by exploring the feature +space using other basis expansions such higher-order polynomials, +wavelets, splines etc. + +
+If our feature space is not easy to separate, as shown in the figure +here, we can achieve a better separation by introducing more complex +basis functions. The ideal would be, as shown in the next figure, to, via a specific transformation to +obtain a separation between the classes which is almost linear. + +
+The change of basis, from \( x\rightarrow z=\phi(x) \) leads to the same type of equations to be solved, except that +we need to introduce for example a polynomial transformation to a two-dimensional training set. + +
+
+
+
+Suppose we define a polynomial transformation of degree two (we continue to live in a plane with \( x_1 \) and \( x_2 \) as variables) +$$ +z = \phi(x) =\left(1, x_1, x_2, x_1^2, x_2^2, x_1x_2). +$$ + +
+With our new basis, the equations we solved earlier are basically the same, that is we have now (without the slack option for simplicity) +$$ +{\cal L}=\sum_i\lambda_i-\frac{1}{2}\sum_{ij}^n\lambda_i\lambda_jy_iy_j\boldsymbol{z}_i^T\boldsymbol{Z}_j, +$$ + +subject to the constraints \( \lambda_i\geq 0 \), \( \sum_i\lambda_iy_i=0 \), and for the support vectors +$$ +y_i(\boldsymbol{w}^T\boldsymbol{z}_i+b)= 1 \hspace{0.1cm}\forall i, +$$ + +from which we also find \( b \). + +
+
+
+
+
+
+
+ + +
+
+
+
+If we use Python as programming language and wish to venture beyond +scikit-learn, tensorflow and similar software which makes our +lives so much easier, we need to dive into the wonderful world of +quadratic programming. We can, if we wish, solve the minimization +problem using say standard gradient methods or conjugate gradient +methods. However, these methods tend to exhibit a rather slow +converge. So, welcome to the promised land of quadratic programming. + +
+The functions we need are contained in the quadratic programming package CVXOPT and we need to import it +
+ + +
import numpy
+import cvxopt
++Let us first set up the standard form the of quadratic programming (QP) equations by defining the problem as +$$ +\mathrm{min} +$$ + +
+subject to Gx u. Note that x itself is not provided to the solver, since it is an internal +variable being optimized over. In particular, this means that the solver has no explicit knowledge +of x itself; everything is implicity defined by the supplied parameters. It is essential +that the same variable order is maintained for the relevant parameters (e.g., qi +Non-convexity implies the existence of local optima, making it difficult to find global optima. + +
+collapsed all inequality constraints into a single G matrix of the standard form. +Since there are no equality constraints, we do not need to provide the empty A, b. Note +that even though y +2 did not appear in the original objective, we had to include it with zero +coefficients in P because the solver parameters must be defined using the full set of variables. +Even if certain variables only appear in constraints, they will still need to be expressed with +zero coefficients in the objective parameters, and vice versa. +Let us first define the above parameters in Python. CVXOPT supplies its own matrix +object; all arguments given to its solvers must be in this matrix type. There are two ways +to do this. The first is to define the matrix directly with (potentially nested) lists: +from cvxopt import matrix +
+ + +
P = matrix([[1.0,0.0],[0.0,0.0]])
+q = matrix([3.0,4.0])
+G = matrix([[-1.0,0.0,-1.0,2.0,3.0],[0.0,-1.0,-3.0,5.0,4.0]])
+h = matrix([0.0,0.0,-15.0,100.0,80.0])
++ diff --git a/doc/pub/svm/html/svm.html b/doc/pub/svm/html/svm.html index 4d0d9e0e2..c9aca2c61 100644 --- a/doc/pub/svm/html/svm.html +++ b/doc/pub/svm/html/svm.html @@ -58,7 +58,12 @@ div { text-align: justify; text-justify: inter-word; } ('The last steps', 2, None, '___sec13'), ('A soft classifier', 2, None, '___sec14'), ('Soft optmization problem', 2, None, '___sec15'), - ('Kernels and non-linearity', 2, None, '___sec16')]} + ('Kernels and non-linearity', 2, None, '___sec16'), + ('The equations', 2, None, '___sec17'), + ('Different kernels', 2, None, '___sec18'), + ('Quadratic coefficient matrix', 2, None, '___sec19'), + ("Mercer's theorem", 2, None, '___sec20'), + ('How do we solve these problems', 2, None, '___sec21')]} end of tocinfo -->
@@ -644,6 +649,120 @@ $$+The cases we have studied till were all characterized by two classes +with a close to linear separability. The classifiers we have described +so far find linear boundaries in our input feature space. It is +possible to make our procedure more flexible by exploring the feature +space using other basis expansions such higher-order polynomials, +wavelets, splines etc. + +
+If our feature space is not easy to separate, as shown in the figure +here, we can achieve a better separation by introducing more complex +basis functions. The ideal would be, as shown in the next figure, to, via a specific transformation to +obtain a separation between the classes which is almost linear. + +
+The change of basis, from \( x\rightarrow z=\phi(x) \) leads to the same type of equations to be solved, except that +we need to introduce for example a polynomial transformation to a two-dimensional training set. + +
+
+
+
+Suppose we define a polynomial transformation of degree two (we continue to live in a plane with \( x_1 \) and \( x_2 \) as variables) +$$ +z = \phi(x) =\left(1, x_1, x_2, x_1^2, x_2^2, x_1x_2). +$$ + +
+With our new basis, the equations we solved earlier are basically the same, that is we have now (without the slack option for simplicity) +$$ +{\cal L}=\sum_i\lambda_i-\frac{1}{2}\sum_{ij}^n\lambda_i\lambda_jy_iy_j\boldsymbol{z}_i^T\boldsymbol{Z}_j, +$$ + +subject to the constraints \( \lambda_i\geq 0 \), \( \sum_i\lambda_iy_i=0 \), and for the support vectors +$$ +y_i(\boldsymbol{w}^T\boldsymbol{z}_i+b)= 1 \hspace{0.1cm}\forall i, +$$ + +from which we also find \( b \). + +
+
+
+
+
+
+
+ + +
+
+
+
+If we use Python as programming language and wish to venture beyond +scikit-learn, tensorflow and similar software which makes our +lives so much easier, we need to dive into the wonderful world of +quadratic programming. We can, if we wish, solve the minimization +problem using say standard gradient methods or conjugate gradient +methods. However, these methods tend to exhibit a rather slow +converge. So, welcome to the promised land of quadratic programming. + +
+The functions we need are contained in the quadratic programming package CVXOPT and we need to import it +
+ + +
import numpy
+import cvxopt
++Let us first set up the standard form the of quadratic programming (QP) equations by defining the problem as +$$ +\mathrm{min} +$$ + +
+subject to Gx u. Note that x itself is not provided to the solver, since it is an internal +variable being optimized over. In particular, this means that the solver has no explicit knowledge +of x itself; everything is implicity defined by the supplied parameters. It is essential +that the same variable order is maintained for the relevant parameters (e.g., qi +Non-convexity implies the existence of local optima, making it difficult to find global optima. + +
+collapsed all inequality constraints into a single G matrix of the standard form. +Since there are no equality constraints, we do not need to provide the empty A, b. Note +that even though y +2 did not appear in the original objective, we had to include it with zero +coefficients in P because the solver parameters must be defined using the full set of variables. +Even if certain variables only appear in constraints, they will still need to be expressed with +zero coefficients in the objective parameters, and vice versa. +Let us first define the above parameters in Python. CVXOPT supplies its own matrix +object; all arguments given to its solvers must be in this matrix type. There are two ways +to do this. The first is to define the matrix directly with (potentially nested) lists: +from cvxopt import matrix +
+ + +
P = matrix([[1.0,0.0],[0.0,0.0]])
+q = matrix([3.0,4.0])
+G = matrix([[-1.0,0.0,-1.0,2.0,3.0],[0.0,-1.0,-3.0,5.0,4.0]])
+h = matrix([0.0,0.0,-15.0,100.0,80.0])
++ diff --git a/doc/pub/svm/ipynb/ipynb-svm-src.tar.gz b/doc/pub/svm/ipynb/ipynb-svm-src.tar.gz index c0c08712c..a514400d1 100644 Binary files a/doc/pub/svm/ipynb/ipynb-svm-src.tar.gz and b/doc/pub/svm/ipynb/ipynb-svm-src.tar.gz differ diff --git a/doc/pub/svm/ipynb/svm.ipynb b/doc/pub/svm/ipynb/svm.ipynb index 346e18da1..cd34fab3c 100644 --- a/doc/pub/svm/ipynb/svm.ipynb +++ b/doc/pub/svm/ipynb/svm.ipynb @@ -1061,7 +1061,158 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "## Kernels and non-linearity" + "## Kernels and non-linearity\n", + "\n", + "The cases we have studied till were all characterized by two classes\n", + "with a close to linear separability. The classifiers we have described\n", + "so far find linear boundaries in our input feature space. It is\n", + "possible to make our procedure more flexible by exploring the feature\n", + "space using other basis expansions such higher-order polynomials,\n", + "wavelets, splines etc.\n", + "\n", + "If our feature space is not easy to separate, as shown in the figure\n", + "here, we can achieve a better separation by introducing more complex\n", + "basis functions. The ideal would be, as shown in the next figure, to, via a specific transformation to \n", + "obtain a separation between the classes which is almost linear. \n", + "\n", + "The change of basis, from $x\\rightarrow z=\\phi(x)$ leads to the same type of equations to be solved, except that\n", + "we need to introduce for example a polynomial transformation to a two-dimensional training set.\n", + "\n", + "## The equations\n", + "\n", + "Suppose we define a polynomial transformation of degree two (we continue to live in a plane with $x_1$ and $x_2$ as variables)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "z = \\phi(x) =\\left(1, x_1, x_2, x_1^2, x_2^2, x_1x_2).\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "With our new basis, the equations we solved earlier are basically the same, that is we have now (without the slack option for simplicity)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "{\\cal L}=\\sum_i\\lambda_i-\\frac{1}{2}\\sum_{ij}^n\\lambda_i\\lambda_jy_iy_j\\boldsymbol{z}_i^T\\boldsymbol{Z}_j,\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "subject to the constraints $\\lambda_i\\geq 0$, $\\sum_i\\lambda_iy_i=0$, and for the support vectors" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "y_i(\\boldsymbol{w}^T\\boldsymbol{z}_i+b)= 1 \\hspace{0.1cm}\\forall i,\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "from which we also find $b$. \n", + "\n", + "## Different kernels\n", + "\n", + "## Quadratic coefficient matrix\n", + "\n", + "\n", + "## Mercer's theorem\n", + "\n", + "## How do we solve these problems\n", + "\n", + "If we use Python as programming language and wish to venture beyond\n", + "**scikit-learn**, **tensorflow** and similar software which makes our\n", + "lives so much easier, we need to dive into the wonderful world of\n", + "quadratic programming. We can, if we wish, solve the minimization\n", + "problem using say standard gradient methods or conjugate gradient\n", + "methods. However, these methods tend to exhibit a rather slow\n", + "converge. So, welcome to the promised land of quadratic programming.\n", + "\n", + "The functions we need are contained in the quadratic programming package **CVXOPT** and we need to import it" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "import numpy\n", + "import cvxopt" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let us first set up the standard form the of quadratic programming (QP) equations by defining the problem as" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "$$\n", + "\\mathrm{min}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "subject to Gx u. Note that x itself is not provided to the solver, since it is an internal\n", + "variable being optimized over. In particular, this means that the solver has no explicit knowledge\n", + "of x itself; everything is implicity defined by the supplied parameters. It is essential\n", + "that the same variable order is maintained for the relevant parameters (e.g., qi\n", + "Non-convexity implies the existence of local optima, making it difficult to find global optima.\n", + "\n", + "collapsed all inequality constraints into a single G matrix of the standard form.\n", + "Since there are no equality constraints, we do not need to provide the empty A, b. Note\n", + "that even though y\n", + "2 did not appear in the original objective, we had to include it with zero\n", + "coefficients in P because the solver parameters must be defined using the full set of variables.\n", + "Even if certain variables only appear in constraints, they will still need to be expressed with\n", + "zero coefficients in the objective parameters, and vice versa.\n", + "Let us first define the above parameters in Python. CVXOPT supplies its own matrix\n", + "object; all arguments given to its solvers must be in this matrix type. There are two ways\n", + "to do this. The first is to define the matrix directly with (potentially nested) lists:\n", + "from cvxopt import matrix" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "P = matrix([[1.0,0.0],[0.0,0.0]])\n", + "q = matrix([3.0,4.0])\n", + "G = matrix([[-1.0,0.0,-1.0,2.0,3.0],[0.0,-1.0,-3.0,5.0,4.0]])\n", + "h = matrix([0.0,0.0,-15.0,100.0,80.0])" ] } ], diff --git a/doc/pub/svm/pdf/svm-minted.pdf b/doc/pub/svm/pdf/svm-minted.pdf index 278550919..2276b146f 100644 Binary files a/doc/pub/svm/pdf/svm-minted.pdf and b/doc/pub/svm/pdf/svm-minted.pdf differ diff --git a/doc/src/SupportVMachines/svm.do.txt b/doc/src/SupportVMachines/svm.do.txt index ba10dabf7..65aa3197d 100644 --- a/doc/src/SupportVMachines/svm.do.txt +++ b/doc/src/SupportVMachines/svm.do.txt @@ -535,3 +535,98 @@ y_i(\bm{w}^T\bm{x}_i+b) -(1-\xi_) \geq 0 \hspace{0.1cm}\forall i. !split ===== Kernels and non-linearity ===== +The cases we have studied till were all characterized by two classes +with a close to linear separability. The classifiers we have described +so far find linear boundaries in our input feature space. It is +possible to make our procedure more flexible by exploring the feature +space using other basis expansions such higher-order polynomials, +wavelets, splines etc. + +If our feature space is not easy to separate, as shown in the figure +here, we can achieve a better separation by introducing more complex +basis functions. The ideal would be, as shown in the next figure, to, via a specific transformation to +obtain a separation between the classes which is almost linear. + +The change of basis, from $x\rightarrow z=\phi(x)$ leads to the same type of equations to be solved, except that +we need to introduce for example a polynomial transformation to a two-dimensional training set. + +!split +===== The equations ===== + +Suppose we define a polynomial transformation of degree two (we continue to live in a plane with $x_1$ and $x_2$ as variables) +!bt +\[ +z = \phi(x) =\left(1, x_1, x_2, x_1^2, x_2^2, x_1x_2). +\] +!et + +With our new basis, the equations we solved earlier are basically the same, that is we have now (without the slack option for simplicity) +!bt +\[ +{\cal L}=\sum_i\lambda_i-\frac{1}{2}\sum_{ij}^n\lambda_i\lambda_jy_iy_j\bm{z}_i^T\bm{Z}_j, +\] +!et +subject to the constraints $\lambda_i\geq 0$, $\sum_i\lambda_iy_i=0$, and for the support vectors +!bt +\[ +y_i(\bm{w}^T\bm{z}_i+b)= 1 \hspace{0.1cm}\forall i, +\] +!et +from which we also find $b$. + +!split +===== Different kernels ===== + +!split +===== Quadratic coefficient matrix ===== + +!split +===== Mercer's theorem ===== + +!split +===== How do we solve these problems ===== + +If we use Python as programming language and wish to venture beyond +_scikit-learn_, _tensorflow_ and similar software which makes our +lives so much easier, we need to dive into the wonderful world of +quadratic programming. We can, if we wish, solve the minimization +problem using say standard gradient methods or conjugate gradient +methods. However, these methods tend to exhibit a rather slow +converge. So, welcome to the promised land of quadratic programming. + +The functions we need are contained in the quadratic programming package _CVXOPT_ and we need to import it +!bc pycod +import numpy +import cvxopt +!ec + +Let us first set up the standard form the of quadratic programming (QP) equations by defining the problem as +!bt +\[ +\mathrm{min} +\] +!et + +subject to Gx u. Note that x itself is not provided to the solver, since it is an internal +variable being optimized over. In particular, this means that the solver has no explicit knowledge +of x itself; everything is implicity defined by the supplied parameters. It is essential +that the same variable order is maintained for the relevant parameters (e.g., qi +Non-convexity implies the existence of local optima, making it difficult to find global optima. + +collapsed all inequality constraints into a single G matrix of the standard form. +Since there are no equality constraints, we do not need to provide the empty A, b. Note +that even though y +2 did not appear in the original objective, we had to include it with zero +coefficients in P because the solver parameters must be defined using the full set of variables. +Even if certain variables only appear in constraints, they will still need to be expressed with +zero coefficients in the objective parameters, and vice versa. +Let us first define the above parameters in Python. CVXOPT supplies its own matrix +object; all arguments given to its solvers must be in this matrix type. There are two ways +to do this. The first is to define the matrix directly with (potentially nested) lists: +from cvxopt import matrix +!bc pycod +P = matrix([[1.0,0.0],[0.0,0.0]]) +q = matrix([3.0,4.0]) +G = matrix([[-1.0,0.0,-1.0,2.0,3.0],[0.0,-1.0,-3.0,5.0,4.0]]) +h = matrix([0.0,0.0,-15.0,100.0,80.0]) +!ec