Public dataset aims to teach AI to 'show its work' on CT scans

Lisa Lock

Scientific Editor

Robert Egan

Senior Editor

NEJM AI(2026). DOI: 10.1056/aidbp2501220

Earlier this summer, an international team led by biomedical informatics researchers at Harvard Medical School released the first public dataset that connects descriptive radiological information, such as "3 mm nodule in the lower left lobe," to precise locations in chest CT scans.

AI models trained on this dataset could analyze CT scans and communicate results more effectively with clinicians, reporting findings in descriptive language and linking those descriptions to specific pixels within three-dimensional scans.

The work, published in NEJM AI, addresses a critical gap in medical AI by making radiology findings clearer and easier to confirm, the researchers said.

"The finding becomes something a clinician can verify instead of something they have to take on faith from an AI tool," said senior author Pranav Rajpurkar, associate professor of biomedical informatics in the Blavatnik Institute at HMS.

The researchers made the dataset, ReXGroundingCT, accessible to the larger scientific community leading up to an official challenge at the International Conference on Medical Image Computing and Computer Assisted Intervention, which concluded on Sept. 27. The challenge drew 53 teams from 15 countries testing different approaches to building models that can link natural language to locations in a CT scan, a process known as grounding.

"This turns grounding from a nice idea into a benchmark people can build against," Rajpurkar said. "The challenge produced state-of-the-art solutions for this task, pushing results well past where the field stood when we started."

Grounding radiology findings

AI models have the potential to help radiologists analyze medical scans quickly and thoroughly, but clinicians need to be able to confirm their results. This is particularly challenging with CT scans, which generate complex 3D images.

Radiologists visualize these images as a series of 2D slices and create reports that describe the locations of anomalies. For an AI model to do the same, it needs to pair clear and accurate descriptions with visual markers that show clinicians exactly where to look across hundreds of 2D slices.

"Datasets like ours will enable training models that can not only output a report but also show you exactly where it thinks these findings exist, so it becomes much easier for the clinician to verify each finding," said Mohammed Baharoon, a Harvard Kenneth C. Griffin Graduate School of Arts and Sciences Ph.D. student in Rajpurkar's lab and first author on the paper.

Applications for medical students and patients

The researchers see additional uses for the dataset in medical education. Rajpurkar and Baharoon previously designed an AI-powered platform for radiology training based on a dataset of annotated X-rays. The platform helps trainees learn to write reports and localize findings.

Over the course of a year, Rajpurkar, Baharoon and their colleagues coordinated with medical annotators, medical students and radiologists to manually annotate more than 16,000 anomalies across 3,142 scans. Each annotation connected language in a report to specific locations within a CT scan and was double-checked by a board-certified radiologist.

"My biggest hope is that this trains a generation of models that don't just describe a finding but actually show where it is," Rajpurkar said. "A model shouldn't only say, "There's a nodule." It should put a marker on the exact voxel it's talking about, so every statement comes with its evidence."

The dataset could be used for tools that improve the patient experience as well, Rajpurkar said. His lab has already built an AI system that generates video explanations of radiology findings to help patients understand their diagnostic results.

With the ReXGroundingCT dataset, the researchers hope to create a training module for CT scans, which are significantly more complicated than X-rays. "Grounding enables tools that show patients exactly where on their own scan a finding sits and translate the surrounding jargon into plain language," he said.

More information

Mohammed Baharoon et al, ReXGroundingCT: A 3D Chest CT Dataset for Segmentation of Findings from Free-Text Reports, NEJM AI (2026). DOI: 10.1056/aidbp2501220

Key medical concepts

Chest Computed Tomography

Clinical categories

Diagnostic radiology

Citation: Public dataset aims to teach AI to 'show its work' on CT scans (2026, October 8) retrieved 8 October 2026 from https://medicalxpress.com/news/2026-10-dataset-aims-ai-ct-scans.html