SORA

Advancing, promoting and sharing knowledge of health through excellence in teaching, clinical practice and research into the prevention and treatment of illness

Minimum sample size for external validation of a clinical prediction model with a continuous outcome.

Archer, L; Snell, KIE; Ensor, J; Hudda, MT; Collins, GS; Riley, RD (2020) Minimum sample size for external validation of a clinical prediction model with a continuous outcome. Stat Med, 40 (1). pp. 133-146. ISSN 1097-0258 https://doi.org/10.1002/sim.8766
SGUL Authors: Hudda, Mohammed Taqui

[img]
Preview
PDF Published Version
Available under License Creative Commons Attribution.

Download (1MB) | Preview

Abstract

Clinical prediction models provide individualized outcome predictions to inform patient counseling and clinical decision making. External validation is the process of examining a prediction model's performance in data independent to that used for model development. Current external validation studies often suffer from small sample sizes, and subsequently imprecise estimates of a model's predictive performance. To address this, we propose how to determine the minimum sample size needed for external validation of a clinical prediction model with a continuous outcome. Four criteria are proposed, that target precise estimates of (i) R2 (the proportion of variance explained), (ii) calibration-in-the-large (agreement between predicted and observed outcome values on average), (iii) calibration slope (agreement between predicted and observed values across the range of predicted values), and (iv) the variance of observed outcome values. Closed-form sample size solutions are derived for each criterion, which require the user to specify anticipated values of the model's performance (in particular R2 ) and the outcome variance in the external validation dataset. A sensible starting point is to base values on those for the model development study, as obtained from the publication or study authors. The largest sample size required to meet all four criteria is the recommended minimum sample size needed in the external validation dataset. The calculations can also be applied to estimate expected precision when an existing dataset with a fixed sample size is available, to help gauge if it is adequate. We illustrate the proposed methods on a case-study predicting fat-free mass in children.

Item Type: Article
Additional Information: © 2020 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd. This is an open access article under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits use, distribution and reproduction in any medium, provided the original work is properly cited.
Keywords: R-squared, calibration, continuous outcomes, external validation, prediction model, sample size, 0104 Statistics, 1117 Public Health and Health Services, Statistics & Probability
SGUL Research Institute / Research Centre: Academic Structure > Population Health Research Institute (INPH)
Journal or Publication Title: Stat Med
ISSN: 1097-0258
Language: eng
Dates:
DateEvent
23 December 2020Published
4 November 2020Published Online
11 September 2020Accepted
Publisher License: Creative Commons: Attribution 4.0
Projects:
Project IDFunderFunder ID
FS/17/76/33286British Heart Foundationhttp://dx.doi.org/10.13039/501100000274
C49297/A27294Cancer Research UKhttp://dx.doi.org/10.13039/501100000289
777090European Horizon 2020UNSPECIFIED
UNSPECIFIEDMedical Research Councilhttp://dx.doi.org/10.13039/501100000265
UNSPECIFIEDNIHR Biomedical Research Centre, OxfordUNSPECIFIED
UNSPECIFIEDNIHR Clinical Trials Unit Support FundingUNSPECIFIED
UNSPECIFIEDNIHR SPCRUNSPECIFIED
390NIHR SPCR Evidence Synthesis Working GroupUNSPECIFIED
102215/2/13/2Wellcome Trusthttp://dx.doi.org/10.13039/100004440
PubMed ID: 33150684
Go to PubMed abstract
URI: https://openaccess.sgul.ac.uk/id/eprint/112607
Publisher's version: https://doi.org/10.1002/sim.8766

Actions (login required)

Edit Item Edit Item