Abstract

BACKGROUND: The global prevalence of diabetes is rising, alongside costs and workload associated with screening for diabetic eye disease (diabetic retinopathy). Automated retinal image analysis systems (ARIAS) could replace primary human grading of images for diabetic retinopathy. We evaluated multiple ARIAS in a real-life screening programme. METHODS: Eight of 25 invited and potentially eligible CE-marked systems for diabetic retinopathy detection from retinal images agreed to participate. From 202 886 screening encounters at the North East London Diabetic Eye Screening Programme (between Jan 1, 2021, and Dec 31, 2022) we curated a database of 1·2 million images and sociodemographic and grading data. Images were manually graded by up to three graders according to a standard national protocol. ARIAS performance overall and by subgroups of age, sex, ethnicity, and index of multiple deprivation (IMD) were assessed against the reference standard, defined as the final human grade in the worst eye for referable diabetic retinopathy (primary outcome). Vendor algorithms did not have access to human grading data. FINDINGS: Sensitivity across vendors ranged from 83·7% to 98·7% for referable diabetic retinopathy, from 96·7% to 99·8% for moderate-to-severe non-proliferative diabetic retinopathy, and from 95·8% to 99·5% for proliferative diabetic retinopathy. Sensitivity was largely consistent for moderate-to-severe non-proliferative and proliferative diabetic retinopathy by subgroups of age, sex, ethnicity, and IMD for all ARIAS. For mild-to-moderate non-proliferative diabetic retinopathy with referable maculopathy, sensitivity across vendors ranged from 79·5% to 98·3%, with greater variability across population subgroups. False positive rates for no observable diabetic retinopathy ranged from 4·3% to 61·4% and within vendors varied by 0·5 to 44 percentage points across population subgroups. INTERPRETATION: ARIAS showed high sensitivity for medium-risk and high-risk diabetic retinopathy in a real-world screening service, with equitable performance across population subgroups. ARIAS could provide a cost-effective solution to deal with the rising burden of screening for diabetic retinopathy by safely triaging for human grading, substantially increasing grading capacity and rapid diabetic retinopathy detection. FUNDING: NHS Transformation Directorate, The Health Foundation, and The Wellcome Trust.