How the Human Touch Boosts AI Surgical Training

Medical students who received real-time, personalized feedback from a human instructor informed by artificial intelligence (AI)–generated error metrics demonstrated significantly superior surgical performance and skill transfer compared with peers who trained with an AI tutor alone.¹

In a virtual neurosurgical simulation, these students achieved higher expertise scores on both routine and complex tasks, lower risk of surgical errors, and improved performance on several safety metrics.

These findings come from a single-blinded, randomized clinical trial conducted at the McGill Neurosurgical Simulation and Artificial Intelligence Learning Centre, published in JAMA Surgery (NCT06273579). The study randomized 88 medical students from 4 Canadian medical schools into 3 groups: group 1 received real-time feedback solely from the AI tutor (the Intelligent Continuous Expertise Monitoring System, or ICEMS), group 2 received human instruction using the same language as the AI, and group 3 received personalized instruction from a human educator who adapted feedback based on AI error signals. All participants used the NeuroVR simulator to perform a series of 6 simulated brain tumor resections followed by 1 realistic task, with intraoperative feedback varying by group.

Researchers emphasized the importance of human input in AI technical skill development. | Image credit: Framestock – stock.adobe.com

In trial 5 of the practice simulations, group 3 outperformed the AI-only control group by a mean composite expertise score difference of 0.26 (95% CI, 0.09-0.43; P = .01). On the final realistic brain tumor resection task, group 3’s expertise scores averaged –0.14 (95% CI, –0.25 to –0.04), compared with –0.35 (95% CI, –0.45 to –0.24) for group 1 and –0.32 (95% CI, –0.43 to –0.21) for group 2. The personalized instruction group also demonstrated significantly lower bleeding risk (mean difference vs group 1, 0.11; 95% CI, 0.05-0.16; P < .001) and injury risk (mean difference vs group 1, 0.03; 95% CI, 0.01-0.04; P = .009).

Group 3 consistently outperformed the other 2 groups across key metrics. Compared with the control group, these students achieved significantly lower bleeding risk from trials 3 to 6 and showed superior performance in aspirator force during trials 3, 4, and 6. While both intervention groups improved over time, only group 3 demonstrated significant gains in both surgical performance and transfer of skill to a more complex task.

The authors emphasized that human-AI collaboration—not AI alone—offers the most promise for technical skill development.

“Consistent with tenets of learning theory, providing human instructors with quantitative AI performance data and allowing them to use their expertise to tailor and contextualize feedback leads to improved learning,” the authors said. “Increased intraoperative educator-student engagement in this learning paradigm based on quantitative learner performance data may be the critical element explaining this study’s findings.”

Interestingly, group 3 also reported higher cognitive load and more frequent negative activating emotions, such as frustration, than their peers—factors often associated with higher mental engagement. While these emotions may impede learning in some contexts, the study authors suggested they might have contributed to better retention and performance in this case.

“This study helps provide pathways toward the overarching goal of creating an intelligent operating room using intraoperative intelligent tutoring systems capable of assessing and training learners while minimizing errors during human surgical procedures,” they wrote.

It’s important to note this study focused on novice learners and not more experienced surgical trainees, warranting further research into how different levels of expertise work alongside AI.

“Understanding how medical students can attain AI-derived benchmarks of more advanced learners has offered insights into the optimization of surgical intelligent tutoring systems,” the authors said. “Although studies involving neurosurgical residents are in preparation, the limited number of available residents may result in an inability to achieve sufficient power to detect statistically significant differences.”

Prior research also supports the integration of human expertise with AI in surgical education. In one randomized trial, ICEMS feedback led to greater learning gains than expert feedback alone, especially when instructors lacked access to real-time AI data.² A separate cohort study highlighted potential drawbacks of AI-only instruction, noting declines in certain efficiency metrics.³ Additionally, a randomized crossover trial showed that ICEMS feedback significantly improved performance when delivered after expert instruction, further suggesting that combining AI and human input may be more effective than using AI in isolation.⁴

References

1. Giglio B, Albeloushi A, Alhaj AK, et al. Artificial intelligence–augmented human instruction and surgical simulation performance a randomized clinical trial. JAMA Surg. Published online August 6, 2025. doi:10.1001/jamasurg.2025.2564
2. Yilmaz R, Bakhaidar M, Alsayegh A, et al. Real-time multifaceted artificial intelligence vs in-person instruction in teaching surgical technical skills: a randomized controlled trial. Sci Rep. 2024;14(1):15130. doi:10.1038/s41598-024-65716-8
3. Fazlollahi AM, Yilmaz R, Winkler-Schwartz A, et al. AI in surgical curriculum design and unintended outcomes for technical competencies in simulation training. JAMA Netw Open. 2023;6(9):e2334658. doi:10.1001/jamanetworkopen.2023.34658
4. Yilmaz R, Fazlollahi A, Alsayegh A, Bakhaidar M, Del Maestro R. 428 Artificial intelligence training versus in-person expert training in teaching simulated tumor resection skills - a cross-over randomized controlled trial. Neurosurgery. 2024;70 (suppl 1):129-130. doi:10.1227/neu.0000000000002809_428