Abstract
Researchers can use the crossvalidated predictive ability test (CVPAT) to evaluate and substantiate the predictive capabilities of their model. In SmartPLS, the CVPAT results are available via the PLSpredict results report.
Description
The crossvalidated predictive ability test (CVPAT) represents an alternative to PLSpredict for predictionoriented assessment of PLSSEM results. The CVPAT was developed by Liengaard et al. (2021) for predictionoriented model comparison in PLSSEM. Sharma et al. (2022) extended the CVPAT for evaluating the model’s predictive capabilities. CVPAT applies an outofsample prediction approach to calculate the model's prediction error, which determines the average loss value. For predictionbased model assessment, this average loss value is compared to the average loss value of a prediction using indicator averages (IA) as a naive benchmark and the average loss value of a linear model (LM) forecast as a more conservative benchmark. PLSSEM’s average loss should be lower than the average loss of the benchmarks which is expressed by a negative difference in the average loss values. CVPAT tests whether PLSSEM’s average loss is significantly lower than the average loss of the benchmarks. Therefore, the difference of the average loss values should be significantly below zero to substantiate better predictive capabilities of the model compared to the prediction benchmarks.
In SmartPLS, the CVPAT results are available via the PLSpredict results report. The IA and LM benchmark results are comparable to the Q² and LM values obtained by PLSpredict (Shmueli et al., 2016, 2019). However, while PLSpredict uses the indicators of the early antecedent constructs for the results computation, CVPAT uses the indicators direct antecedents (for further details on early and direct antecedents, see Danks, 2021).
The outofsample predictions used in CVPAT assist researchers in evaluating the predictive capabilities of their model. Therefore, CVPAT should be included in the evaluation of PLSSEM results (Hair et al., 2022).
Additional procedures and extensions are under development and may become part of future SmartPLS versions (e.g., the CVPATbased model comparison).
PLSpredict Settings for obtaining CVPAT results in SmartPLS
Number of Folds
Default: 10
In kfold crossvalidation the algorithm splits the full dataset into k equally sized subsets of data. The algorithm then predicts each fold (holdout sample) with the remaining k1 subsets, which, in combination, become the training sample. For example, when k equals 10 (i.e., 10folds), a dataset of 200 observations will be split into 10 subsets with 20 observations per subset. The algorithm then predicts ten times each fold with the nine remaining subsets.
Number of Repetitions
Default: 10
The number of repetitions indicates how often PLSpredict algorithm runs the kfold cross validation on random splits of the full dataset into k folds.
Traditionally, crossvalidation only uses one random split into kfolds. However, a single random split can make the predictions strongly dependent on this random assignment of data (observations) into the kfolds. Due to the random partition of data, executions of the algorithm at different points of time may vary in their predictive performance results.
Repeating the kfold crossvalidation with different random data partitions and computing the average across the repetitions ensures a more stable estimate of the predictive performance of the PLS path model.
References

Danks, N. (2021). The Piggy in the Middle: The Role of Mediators in PLSSEMbased Prediction. ACM SIGMIS Database: the DATABASE for Advances in Information Systems, 52, 2442.

Hair, J. F., Hult, G. T. M., Ringle, C. M., & Sarstedt, M. (2022). A Primer on Partial Least Squares Structural Equation Modeling (PLSSEM) (3 ed.). Thousand Oaks, CA: Sage.

Liengaard, B. D., Sharma, P. N., Hult, G. T. M., Jensen, M. B., Sarstedt, M., Hair, J. F., & Ringle, C. M. (2021). Prediction: Coveted, Yet Forsaken? Introducing a Crossvalidated Predictive Ability Test in Partial Least Squares Path Modeling. Decision Sciences, 52(2), 362392.

Sharma, P. N., Liengaard, B. D., Hair, J. F., Sarstedt, M., & Ringle, C. M. (2023). Predictive Model Assessment and Selection in Compositebased Modeling Using PLSSEM: Extensions and Guidelines for Using CVPAT. European Journal of Marketing, 57(6), 16621677.

Shmueli, G., Ray, S., Estrada, J. M. V., & Chatla, S. B. (2016). The Elephant in the Room: Predictive Performance of PLS Models. Journal of Business Research, 69(10), 45524564.

Shmueli, G., Sarstedt, M., Hair, J. F., Cheah, J. H., Ting, H., Vaithilingam, S., & Ringle, C. M. (2019). Predictive Model Assessment in PLSSEM: Guidelines for Using PLSpredict. European Journal of Marketing, 53(11), 23222347.
Please always cite the use of SmartPLS!
Ringle, Christian M., Wende, Sven, & Becker, JanMichael. (2024). SmartPLS 4. Bönningstedt: SmartPLS. Retrieved from https://www.smartpls.com