Before moving to the logistic model, let us try to use our linear regression model to classify these two outcomes. We could for example fit a linear model to the default case if \( y_i > 0.5 \) and the no default case \( y_i \leq 0.5 \).
We would then have our weighted linear combination, namely $$ \begin{equation} \hat{y} = \hat{X}^T\hat{\beta} + \hat{\epsilon}, \tag{1} \end{equation} $$ where \( \hat{y} \) is a vector representing the possible outcomes, \( \hat{X} \) is our \( n\times p \) design matrix and \( \hat{\beta} \) represents our estimators/predictors.