Purpose

The prediction-oriented model selection criteria stem from information theory and have been introduced into the partial least squares structural equation modeling (PLS‐SEM) context by Sharma et al. (2019, 2021). These model selection criteria help researchers to select the best predictive model from a pre-determined range of alternative model set-ups. They, thereby, allow researchers to fully exploit the predictive capabilities of PLS‐SEM.

Note: As an extension, Liengaard et al. (2020) proposed the cross-validated predictive ability test (CVPAT) for predictive model comparison in PLS-SEM. However, this extension has not been implemented in SmartPLS so far (a(and will be the subject of future SmartPLS add-ons). So far, the CVPAT for the predictive model assessment, as proposed by Sharma et al. (2023) has been implemented in the PLSpredict results report.

Criteria

Sharma et al. (2019, 2021) show by means of a Monte Carlo simulation that the in‐sample model selection Bayesian information criterion (BIC) and the Geweke–Meese (GM) criterion are useful substitutes for out‐of‐sample model selection criteria (e.g., RMSA and MAD). SmartPLS provides results of the BIC for model selection. SmartPLS does not include the GM, which is based on the model-complexity adjusted mean square error (MSE) from the saturated (full) model. In PLS-SEM, defining the saturated model is not always straightforward. Especially in advanced modeling situations (e.g. moderation and second-order models) expert judgement is needed to define the saturated model, while an automatic generation of a saturated model could produce strange results.

Note: You can download and run an Excel file to compute the model selection criteria by yourself, including the GM.

Interpretation

Sharma et al.’s (2019, 2021) Monte Carlo simulations show that the BIC and GM are particularly suitable for model comparison tasks. These criteria tend to select the best model among a set of competing models. In addition, both criteria achieve a sound balance between theoretical consistency and high predictive power, even in the absence of a holdout sample. Researchers need to compare BIC and GM values for alternative model set-ups and select the model, which minimizes BIC and GM values. Hence, when using SmartPLS, one would select the model alternative with the lowest BIC outcome.

References

• Liengaard, B. D., Sharma, P. N., Hult, G. T. M., Jensen, M. B., Sarstedt, M., Hair, J. F., & Ringle, C. M. (2021). Prediction: Coveted, Yet Forsaken? Introducing a Cross-validated Predictive Ability Test in Partial Least Squares Path Modeling. Decision Sciences, 52(2), 362-392.

• Sharma, P. N., Liengaard, B. D., Hair, J. F., Sarstedt, M., & Ringle, C. M. (2022). Predictive Model Assessment and Selection in Composite-based Modeling Using PLS-SEM: Extensions and Guidelines for Using CVPAT. European Journal of Marketing, 57(6), 1662-1677.

• Sharma, P. N., Sarstedt, M., Shmueli, G., Kim, K.H, and Thiele, K. O. (2019). PLS-Based Model Selection: The Role of Alternative Explanations in Information Systems Research, Journal of the Association for Information Systems, 20(4), 346-397.

• Sharma, P. N., Shmueli, G., Sarstedt, M., Danks, N., and Ray, S. (2021). Prediction-oriented Model Selection in Partial Least Squares Path Modeling, Decision Sciences, 52(3), 567-607.

• More literature ...