249: Cytopathology AI: Crowded-Cell Gaps and LLM Guardrails cover art

249: Cytopathology AI: Crowded-Cell Gaps and LLM Guardrails

249: Cytopathology AI: Crowded-Cell Gaps and LLM Guardrails

Listen for free

View show details
Send us Fan MailWhat happens when AI models that perform almost perfectly on scattered cervical cells become less reliable than a coin flip on crowded cell groups?In DigiPath Digest #51, I examine what this performance gap tells us about artificial intelligence in cytopathology.The first paper evaluated six convolutional neural network models trained to distinguish benign from high-grade lesions using scattered cervical cytology cells. The models achieved AUCs ranging from 0.950 to 0.996 on the original images.When the same models were applied to hyperchromatic crowded cell groups without retraining or threshold recalibration, their AUCs fell to 0.385-0.683. Different architectures also failed differently. Some overcalled benign clusters, while others missed high-grade clusters.This isn’t simply a technical problem. It illustrates a practical rule for pathology AI: a model should only be trusted for the morphology, specimen type, imaging system, and intended use on which it has been directly validated.I also review the current evidence for large language models in cytopathology. Potential applications include structured reporting, diagnostic support, quality control, education, research, and workflow integration.Some early results appear promising, but each comes with important limitations. One structured-reporting application achieved 99.4% accuracy at a single institution. A diagnostic-support model included the correct answer among its top 10 differentials in 59.1% of general medicine cases. A hybrid quality-control system flagged 84% of errors associated with amended reports, but its false-positive rate wasn’t reported.Most importantly, the review found no language model specifically trained and clinically validated on cytopathology reports.The takeaway is straightforward: we’re still working with narrow AI. Strong performance in one setting doesn’t guarantee performance when the cells, preparation, scanner, institution, or clinical task changes.Low-risk applications may offer the most practical starting point. Text extraction, completeness checks, report consistency review, and quality-control flagging could reduce repetitive work without asking an unvalidated model to make the final diagnosis.Highlights with timestamps00:00 - Welcome to DigiPath Digest #51 and the new lunch-and-learn time01:40 - How two image models performed worse than a coin flip on cell clusters02:29 - Image models, language models, and vision-language models04:40 - Why AI adoption in cytopathology remains low05:25 - Scattered single cells versus hyperchromatic crowded cell groups08:14 - AUCs fall from 0.950-0.996 to 0.385-0.68310:02 - How ResNet-50 and GoogLeNet failed differently11:04 - What the attention maps revealed13:22 - The intended-use lesson for pathology AI17:02 - Current applications of large language models in cytopathology18:39 - Structured reporting and the 99.4% accuracy result19:32 - Diagnostic support, AMIE, and the top-10 limitation21:09 - Quality-control applications and the missing false-positive rate23:30 - Why cytopathology still needs domain-specific language models24:54 - Retrieval-augmented generation, education, and research support27:07 - Two AI families, one requirement: direct validation28:35 - Context of use and intended-use validation29:12 - Human-reviewed training data and destructive book scanning32:02 - FDA and European approaches to evolving AI models35:00 - Protecting patient and practitioner well-being36:21 - Cytopathology-specific benchmarks and shared test sets38:16 - Why low-risk AI applications should come first39:50 - Cytopathology’s direct-to-digital advantage40:39 - Digital Pathology 101 and Pathology VisionsResources from this episodeWatch DigiPath Digest #51Diagnostic performance of AI models trained on scattered single-cell imagesLeveraging large language models to enhance cytopathologyFDA guidance on AI and context of useFDA guidance on predetermined change control plansGoogle Research: AMIE diagnostic medical AIEU In Vitro Diagnostic Medical Devices RegulationEU Artificial Intelligence ActDigital Pathology 101Pathology Visions 2026 Support the showGet the "Digital Pathology 101" FREE E-book and join us!
adbl_web_anon_alc_button_suppression_t1
No reviews yet