The penalization factor \( \lambda \) is inverted in the case of the logistic regression model we use. We will explore several values of \( \lambda \) using both L1 and L2 penalization. We do this using a grid search over different parameters and run a 3-fold cross validation for each configuration. In other words, we fit a model 3 times for each configuration of the hyper parameters.
lambdas = np.logspace(-7, -1, 7)
param_grid = {
"C": list(1.0/lambdas),
"penalty": ["l1", "l2"]
}
clf = skms.GridSearchCV(
skl.LogisticRegression(),
param_grid=param_grid,
n_jobs=-1,
return_train_score=True
)
t0 = time.time()
clf.fit(X_train, y_train)
t1 = time.time()
print (
"Time spent fitting GridSearchCV(LogisticRegression): {0:.3f} sec".format(
t1 - t0
)
)
We can see that logistic regression is quite slow and using the grid search and cross validation results in quite a heavy computation. Below we show the results of the different configurations.
logreg_df = pd.DataFrame(clf.cv_results_)
display(logreg_df)