pod.select_cv_winner

select_cv_winner(cv_scores, cv_se=None)

Picks the expectation model using the one-standard-error rule.

[M25] Sec. 4b(i) (Fig. 5) chose the 3rd-order polynomial over the 4th- and 5th-order polynomials and Kriging, whose 10-fold CV errors were almost identical, “since it is the simplest one among those with smaller errors”; [M25-code] records the standard error of each CV estimate (std(fold MSEs) / sqrt(k)). [digiqual] turns that judgement into a fixed rule: find the model with the lowest mean CV MSE, then choose the simplest model whose CV MSE is no more than one standard error above it. Complexity order is Polynomial 1, 2, …, then Kriging (the most flexible).

Parameters

Name Type Description Default
cv_scores dict (model_type, params) -> mean CV MSE. required
cv_se dict (model_type, params) -> standard error of that CV MSE. Without it, the strict minimum is returned. None

Returns

Name Type Description
tuple tuple[str, Any] The key of the selected model.