Back to 2026 Abstracts
Surgeon–Model Risk Assessment Discordance and Postoperative Outcomes
Sofia Volpi1, Jayson Marwaha2, Margaret Berrigan1, Zeynep Bilgili1, Anupamaa Seshadri1, Daniel Bent1, Stephen Odom1, Charles Parsons1, Alok Gupta1, Charles Cook1, Gabriel Brat1,3 1. Department of Surgery, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA 2. Department of Surgery, University of Michigan, MI 3. Department of Biomedical Informatics, Harvard Medical School, Boston, MA
Introduction: Surgical risk prediction models are often met with skepticism by surgeons, possibly reflecting uncertainty about how well they apply to individual patients. We examined postoperative outcomes in cases where surgeon risk assessment diverged from model predictions. Methods: From 2022-2024, surgeons at a single academic medical center were prospectively provided risk estimates for any 30-day postoperative complication defined by NSQIP, generated using the NSQIP and POTTER models. In real time, surgeons indicated whether individual patient risks were lower, equal to, or higher than model estimates. Discrimination was assessed using area-under-the-curve (AUC) for each model alone and after incorporating surgeon assessment. Patients were stratified into low- and high-risk groups based on model-predicted risk. We evaluated 30-day complications among low-risk patients whom surgeons up-classified, and high-risk patients whom surgeons down-classified. Results: Twenty-two surgeons provided assessments for 1,012 cases (519 low-predicted-risk; 493 high-predicted-risk). Overall, 323 patients (31.9%) developed complications (46.7% high-predicted-risk vs 17.9% low-predicted-risk). POTTER discrimination was 0.82 (0.78-0.86) and NSQIP 0.66 (0.61-0.72), with minimal improvement after incorporating surgeon assessment: 0.84 (0.80-0.87) and 0.69 (0.64–0.74), respectively. However, surgeons up-classified 31.1% of low-predicted-risk cases and 28.0% of these patients developed complications, accounting for 48.0% of all complications within the low-risk group, and including 41 (25.5%) major complications. Conversely, surgeons down-classified 14.4% of high-predicted-risk cases; 40.9% developed complications, including 17.0% with multiple complications and 3 deaths (4.2%). Conclusion: Surgeon prediction can appropriately identify higher-risk patients sometimes missed by models, but can also underestimate some high-risk signals associated with adverse outcomes, limiting overall improvement in discrimination.
Back to 2026 Abstracts
