Which statement best describes the purpose of information criteria such as AIC and BIC in regression modeling?

Which statement best describes the purpose of information criteria such as AIC and BIC in regression modeling?

• A. They measure residual normality
• B. They quantify predictive accuracy on training data only
• C. They assess multicollinearity
• D. They balance model fit with model complexity to avoid overfitting

Answer: (D)

Balancing model fit with model complexity to avoid overfitting is what information criteria aim to do. AIC and BIC combine how well the model fits the data (via the likelihood) with a penalty for the number of estimated parameters. This penalty discourages adding parameters that don’t meaningfully improve fit, promoting a model that generalizes better to new data. They aren’t direct checks of residual normality, they don’t measure multicollinearity, and they aren’t about predictive accuracy only on the training set. The goal is to pick the model with the best trade-off, usually the one with the smallest AIC or BIC, with BIC tending to favor simpler models as sample size grows.

Balancing model fit with model complexity to avoid overfitting is what information criteria aim to do. AIC and BIC combine how well the model fits the data (via the likelihood) with a penalty for the number of estimated parameters. This penalty discourages adding parameters that don’t meaningfully improve fit, promoting a model that generalizes better to new data. They aren’t direct checks of residual normality, they don’t measure multicollinearity, and they aren’t about predictive accuracy only on the training set. The goal is to pick the model with the best trade-off, usually the one with the smallest AIC or BIC, with BIC tending to favor simpler models as sample size grows.