Automated and Artificial Intelligence (AI)-derived performance assessment in surgical simulation: a systematic review
Khalifa, Ahmad Albazooni, Mohammed Saeed, Mohammed Hamdi, Ruha Stanners, Megan Malik, Amman Malik, Adnan
Khalifa, Ahmad
Albazooni, Mohammed
Saeed, Mohammed
Hamdi, Ruha
Stanners, Megan
Malik, Amman
Malik, Adnan
Abstract
Artificial intelligence (AI)-assisted and automated performance assessment is increasingly being incorporated into surgical education, yet the degree, efficacy, and trustworthiness of this performance assessment are unknown. A systematic review of the literature published between 2010 and 2025 on PubMed, Scopus, Embase, and IEEE Xplore (including conference proceedings and gray literature) was completed to identify experimental and observational studies that reported the use of algorithms or automated methods of assessment in technical skill assessment in both simulator-based training and real clinical practice. Information was extracted on study characteristics, algorithm type, task complexity, performance measures, evidence of validity and reliability, and the quality of studies, assessed using relevant tools, with results reported descriptively and trends reviewed by study type, year, country, and task domain. Twenty-nine studies met the inclusion criteria, with most using supervised learning algorithms to evaluate technical skills; performance measures ranged widely, and studies were inconsistent in documenting the validity and reliability of the assessments. Few studies presented a real-time adaptive feedback system. The literature reflects a shift toward simulation-based assessment and the increasing use of multimodal data sources. Still, methodological heterogeneity, poor transparency, and a lack of validity and reliability prevent generalisability across tasks and contexts. AI-based assessment has potential in surgical education, with real-time adaptive assessment particularly interesting; however, standardised, validated methods are required to ensure reproducible measures of performance and ethical implementation in surgical simulation training.
MIDER Authors
Affiliations
University Hospitals Birmingham NHS Foundation Trust; Sandwell and West Birmingham NHS Trust; Tameside and Glossop Integrated Care NHS Foundation Trust; Dudley Group NHS Foundation Trust; et al.
Date
2025-12-31
Type
Article
Collections
Citation
Khalifa A, Tahhan O, Albazooni M, Saeed M, Hamdi R, Stanners M, Malik A, Malik A. Automated and Artificial Intelligence (AI)-Derived Performance Assessment in Surgical Simulation: A Systematic Review. Cureus. 2025 Dec 31;17(12):e100477. doi: 10.7759/cureus.100477