Archer, L; Snell, KIE; Ensor, J; Hudda, MT; Collins, GS; Riley, RD
(2020)
Minimum sample size for external validation of a clinical prediction model with a continuous outcome.
Stat Med, 40 (1).
pp. 133-146.
ISSN 1097-0258
https://doi.org/10.1002/sim.8766
SGUL Authors: Hudda, Mohammed Taqui
Abstract
Clinical prediction models provide individualized outcome predictions to inform patient counseling and clinical decision making. External validation is the process of examining a prediction model's performance in data independent to that used for model development. Current external validation studies often suffer from small sample sizes, and subsequently imprecise estimates of a model's predictive performance. To address this, we propose how to determine the minimum sample size needed for external validation of a clinical prediction model with a continuous outcome. Four criteria are proposed, that target precise estimates of (i) R2 (the proportion of variance explained), (ii) calibration-in-the-large (agreement between predicted and observed outcome values on average), (iii) calibration slope (agreement between predicted and observed values across the range of predicted values), and (iv) the variance of observed outcome values. Closed-form sample size solutions are derived for each criterion, which require the user to specify anticipated values of the model's performance (in particular R2 ) and the outcome variance in the external validation dataset. A sensible starting point is to base values on those for the model development study, as obtained from the publication or study authors. The largest sample size required to meet all four criteria is the recommended minimum sample size needed in the external validation dataset. The calculations can also be applied to estimate expected precision when an existing dataset with a fixed sample size is available, to help gauge if it is adequate. We illustrate the proposed methods on a case-study predicting fat-free mass in children.
Item Type: |
Article
|
Additional Information: |
© 2020 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.
This is an open access article under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits use, distribution and reproduction in any medium, provided the original work is properly cited. |
Keywords: |
R-squared, calibration, continuous outcomes, external validation, prediction model, sample size, 0104 Statistics, 1117 Public Health and Health Services, Statistics & Probability |
SGUL Research Institute / Research Centre: |
Academic Structure > Population Health Research Institute (INPH) |
Journal or Publication Title: |
Stat Med |
ISSN: |
1097-0258 |
Language: |
eng |
Dates: |
Date | Event |
---|
23 December 2020 | Published | 4 November 2020 | Published Online | 11 September 2020 | Accepted |
|
Publisher License: |
Creative Commons: Attribution 4.0 |
Projects: |
|
PubMed ID: |
33150684 |
![](http://static.pubmed.gov/portal/portal3rc.fcgi/2745237/img/27842) |
Go to PubMed abstract |
URI: |
https://openaccess.sgul.ac.uk/id/eprint/112607 |
Publisher's version: |
https://doi.org/10.1002/sim.8766 |
Statistics
Item downloaded times since 17 Nov 2020.
Actions (login required)
![Edit Item](/style/images/action_view.png) |
Edit Item |