AI Chatbots Score Below 28% on Colorectal Cancer Questions

Six leading artificial intelligence (AI) chatbots each fell far short of a passing grade on a colorectal cancer (CRC) knowledge test when barred from explaining their reasoning, with the top performer answering just 27.7% of questions correctly.¹ The findings, from a comparative pilot study published in Updates in Surgery, add fresh evidence that headline benchmark scores can mask serious reliability gaps when large language models (LLMs) are used the way busy clinicians actually use them: fast, and without a prompt to think it through.

The researchers administered 137 text-based multiple-choice questions, adapted from the 2023 edition of the Chinese Colorectal Cancer Diagnosis and Treatment Guidelines and spanning 12 clinical modules, to Gemini 3 Pro Preview, GPT-5.1, Kimi K2 Thinking, DeepSeek V3.2, Qwen3-Max, and Claude Opus 4.5. Each model received a "zero-shot" constrained prompt instructing it to output only the correct answer letter, with no rationale.

Why the Chatbots Scored So Low

Accuracy clustered at the bottom of the scale. Kimi K2 Thinking led at 27.7%, followed by Claude Opus 4.5 at 26.3%, Gemini 3 Pro Preview at 16.1%, DeepSeek V3.2 at 15.3%, GPT-5.1 at 14.6%, and Qwen3-Max at 13.9%. Statistical testing found no significant difference among the 6, suggesting the models share a common ceiling when forced to retrieve answers without a reasoning step regardless of architecture. 

Performance also swung widely by topic. On endoscopic and imaging questions, Kimi K2 Thinking reached 37% while Qwen3-Max managed just 7.4%. Universal poor performance was observed in overview and screening, where while most models failed to clear 30%, Kimi K2 Thinking was an outlier at 50%. Furthermore, no model exceeded 20% on local recurrence and special populations, a domain that requires integrating several guideline recommendations at once.

The authors argue this pattern reflects "pattern matching" rather than genuine clinical reasoning. In one representative error, models named the more commonly cited long-course radiotherapy dose even when the question specified a short-course regimen. In another, they confused the surgical margin of the rectal mesentery with the intestinal wall margin, producing an unsafe value.

What Drove the Wrong Answers

A qualitative review sorted the failures into 3 buckets: fact retrieval errors (65%), logic or reasoning errors (25%), and outright hallucinations (10%). Although hallucinations were least common, the authors flagged them as the most dangerous. One example had models recommending a tumor marker for all patients with CRC rather than only those with suspected peritoneal or ovarian metastasis, a mistake that could drive unnecessary testing and cost.

The concern echoes earlier reporting by The American Journal of Managed Care® (AJMC®) on the limits of AI in oncology. Ryan Nguyen, DO, of the University of Illinois Chicago, described research in which ChatGPT largely reproduced National Comprehensive Cancer Network (NCCN) recommendations for non–small cell lung cancer but hallucinated treatment combinations 30% to 40% of the time, warning that precision oncology leaves no room for a wrong answer.² The pattern also aligns with a recent study covered by AJMC finding that AI chatbots produced unsafe or clinically inappropriate answers to nearly one-third of patient-posed medical questions.³

Why Difficulty Didn't Predict Accuracy

Perhaps the most unsettling finding was the absence of any correlation between question difficulty and accuracy.¹ The models missed "easy" items as often as hard ones, behaving, in the authors' framing, like "stochastic parrots" generating plausible text rather than applying clinical logic. 

The study also reinforces why prompt design matters. Prior work cited by the authors suggests chain-of-thought (CoT) prompting can lift medical AI accuracy by 15% to 20%. Stripping that self-explanation away is precisely what exposed the models' unreliable recall. The result is less a verdict on any single model than a warning about a common user behavior of asking for a fast, unexplained answer under time pressure.

What This Means for Managed Care

For health systems weighing AI copilots for clinical decision support, the takeaways are practical. The authors recommend treating current chatbots as "preliminary search engines" rather than "digital professors," pairing them with CoT prompting, human expert verification, and domain-specific fine-tuning before any unsupervised use. That mirrors a broader industry pivot toward guideline-grounded tools, such as the American Society of Clinical Oncology Guidelines Assistant launched in 2025 to deliver cited, guideline-based answers.³

The authors caution that their findings are a snapshot.¹ The sample was small (137 items), the format excluded visual diagnostic tasks, the zero-shot design was deliberately a stress test, and the closed-source models tested may improve with targeted training. Even so, the message for managed care stakeholders is that validation in a specialized domain like oncology demands more than general benchmark performance—and that guardrails, not raw capability, should govern deployment.

References

1. Chen H, Tan X, Wu D, Kang L. Performance of next-generation AI chatbots in colorectal cancer knowledge assessment: a comparative pilot study of ChatGPT-5.1, Gemini-3Pro Preview, DeepSeek-V3.2, Kimi K2 Thinking, Qwen3-Max and Claude Opus 4.5. Updates Surg. 2026. https://doi.org/10.1007/s13304-026-02766-9
2. Jeremias S. AI in oncology: opportunities and challenges for NSCLC. AJMC. January 1, 2025. Accessed July 9, 2026. https://www.ajmc.com/view/ai-in-oncology-opportunities-and-challenges-for-nsclc
3. Flinn R. Evidence-Based Oncology. Five ways AI is transforming cancer care—and companies that are making it happen. AJMC. November 11, 2025. Accessed July 9, 2026. https://www.ajmc.com/view/five-ways-ai-is-transforming-cancer-care-and-companies-that-are-making-it-happen