Estimating errors

Having two sides of an equation as equal as possible, means that the absolute or squared difference between the sides must be as close to zero as small. In this case, the difference squared is an appropiate measurement of how errorneous the trial solution is with respect to \( P \) of the neural network. Therefore, the problem our network must solve, is $$ \min_{P}\Big\{ \big(g_t'(x, P) - ( -\gamma g_t(x, P) \big)^2 \Big\} $$

or, in terms of weights and biases for each layer: $$ \min_{P_{\mathrm{hidden} }, \ P_{\mathrm{output} }}\Big\{ \big(g_t'(x, \{ P_{\mathrm{hidden} }, P_{\mathrm{output} }\}) - ( -\gamma g_t(x, \{ P_{\mathrm{hidden} }, P_{\mathrm{output} }\}) \big)^2 \Big\} $$

for an input value \( x \). If the neural network evaluates \( g_t(x, P) \) at more avalues for \( x \), say \( N \) values \( x_i \) for \( i = 1, \dots, N \), then the total error to minimize is $$ \begin{equation} \tag{19} \min_{P}\Big\{\sum_i \big(g_t'(x_i, P) - ( -\gamma g_t(x_i, P) \big)^2 \Big\} \end{equation} $$

Letting \( c(x, P) = \sum_i \big(g_t'(x_i, P) - ( -\gamma g_t(x_i, P) \big)^2 \) denote the cost function, the minimization problem of which our network must solve, is $$ \min_{P} c(x, P) $$

or in terms of \( P_{\mathrm{hidden} } \) and \( P_{\mathrm{output} } \) $$ \min_{P_{\mathrm{hidden} }, \ P_{\mathrm{output} }} c(x, \{P_{\mathrm{hidden} }, P_{\mathrm{output} }\}) $$