The feedforward step is similar to as for the neural netowork, but now considering more than one hidden layer.
The \( i \)-th neuron at layer \( l \) recieves the result \( \vec{x}_j^{(l-1),\text{hidden} } \) from the \( j \)-th neuron at layer \( l-1 \). The \( i \)-th neuron at layer \( l \) weights all of the elements in \( \vec{x}_j^{(l-1),\text{hidden} } \) with a weight vector \( \vec w_{i,j}^{(l), \ \text{hidden} } \) with as many weigths as there are elements in$\vec{x}_j^{(l-1),\text{hidden} }$, and adds a bias \( b_i^{(l), \ \text{hidden} } \): $$ \begin{aligned} z_{i,j}^{(l),\ \text{hidden}} &= b_i^{(l), \ \text{hidden}} + \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T\vec{x}_j^{(l-1),\text{hidden} } \\ &= \begin{pmatrix} b_i^{(l), \ \text{hidden}} & \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T \end{pmatrix} \begin{pmatrix} 1 \\ \vec{x}_j^{(l-1),\text{hidden} } \end{pmatrix} \end{aligned} $$
The output from the \( i \)-th neuron at the hidden layer \( l \) becomes a vector \( \vec{z}_{i}^{(l),\ \text{hidden}} \): $$ \begin{aligned} \vec{z}_{i}^{(l),\ \text{hidden}} &= \Big( b_i^{(l), \ \text{hidden}} + \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T\vec{x}_1^{(l-1),\text{hidden} }, \ \dots \ , \ b_i^{(l), \ \text{hidden}} + \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T\vec{x}_{N_{hidden}^{(l-1)}}^{(l-1),\text{hidden} } \Big) \\ &= \begin{pmatrix} b_i^{(l), \ \text{hidden}} & \big(\vec{w}_{i}^{(l), \ \text{hidden}}\big)^T \end{pmatrix} \begin{pmatrix} 1 & 1 & \dots & 1 \\ \vec{x}_{1}^{(l-1),\text{hidden} } & \vec{x}_{2}^{(l-1),\text{hidden} } & \dots & \vec{x}_{N_{hidden}^{(l-1)}}^{(l-1),\text{hidden} } \end{pmatrix} \end{aligned} $$