Prediction-oriented model selection

Purpose

The prediction-oriented model selection criteria stem from information theory and have been introduced into the partial least squares structural equation modeling (PLS‐SEM) context by Sharma et al. (2019a,b). These model selection criteria help researchers to select the best predictive model from a pre-determined range of alternative model set-ups. They, thereby, allow researchers to fully exploit the predictive capabilities of PLS‐SEM.

Criteria

Sharma et al. (2019a,b) show by means of a Monte Carlo simulation that the in‐sample model selection Bayesian information criterion (BIC) and the Geweke–Meese (GM) criterion are useful substitutes for out‐of‐sample model selection criteria (e.g., RMSA and MAD). SmartPLS provides results of the BIC for model selection. SmartPLS does not include the GM, which is based on the model-complexity adjusted mean square error (MSE) from the saturated (full) model. In PLS-SEM, defining the saturated model is not always straightforward. Especially in advanced modeling situations (e.g. moderation and second-order models) expert judgement is needed to define the saturated model, while an automatic generation of a saturated model could produce strange results.

Note: You can download and run an Excel file to compute the model selection criteria by yourself, including the GM.

Interpretation

Sharma et al.’s (2019a,b) Monte Carlo simulations show that the BIC and GM are particularly suitable for model comparison tasks. These criteria tend to select the best model among a set of competing models. In addition, both criteria achieve a sound balance between theoretical consistency and high predictive power, even in the absence of a holdout sample. Researchers need to compare BIC and GM values for alternative model set-ups and select the model, which minimizes BIC and GM values. Hence, when using SmartPLS, one would select the model alternative with the lowest BIC outcome.

References

Please always cite the use of SmartPLS!

Ringle, Christian M., Wende, Sven, & Becker, Jan-Michael. (2015). SmartPLS 3. Boenningstedt: SmartPLS. Retrieved from https://www.smartpls.com