Study cautions doctors on AI‑generated chest X‑ray highlights

Findings

Doctors should not depend solely on artificial‑intelligence outputs when reading chest X‑rays, a new audit of four vision‑language systems shows. Researchers at the Language Technologies Research Centre of the International Institute of Information Technology Hyderabad tested how well the AI‑produced attention maps matched the disease zones marked by radiologists. They discovered frequent mismatches; an AI model could point to a region that appeared correct yet not correspond to the radiologist’s assessment. The team, led by Parameswari Krishnamurthy with principal investigator Syed Faizan, asked whether the highlighted zones truly reflected the models’ ability to locate disease. They found that some systems first arrived at a diagnostic label and then generated a map based on that label, rather than locating the abnormality directly from the image. When the researchers stripped the diagnostic hint from the input, the models’ localisation performance fell sharply, confirming reliance on diagnosis cues. Different models also varied in how they fared in the technical audit versus radiologist evaluation, underscoring that a tight focus on a suspected spot may omit information about disease spread that clinicians need.

Implications

The work fills a gap left by earlier studies that measured only prediction accuracy. It highlights that AI tools must be judged from a clinical viewpoint and used as assistants, not replacements, for human judgement.

Context

Vision‑language models can be sensitive to how a medical question is phrased, affecting their answers. The researchers are also testing this “paraphrase robustness” to ensure consistent performance across varied clinician language.