Using AI to help physicians diagnose rare genetic diseases affecting children
Key Points
- o3 Deep Research reanalyzed 376 unsolved pediatric cases
- 18 additional diagnoses established (4.8% yield)
- Model produced evidence-linked hypotheses, not clinical decisions
Summary
Published 2026-06-18 in NEJM AI, researchers from Boston Children’s Manton Center, Harvard, and OpenAI used the OpenAI o3 Deep Research reasoning model to retrospectively reanalyze 376 previously unsolved pediatric cases. The model produced evidence-linked, explainable hypotheses that expert reviewers evaluated using standard clinical frameworks; after confirmatory testing and CLIA lab confirmation, 18 additional diagnoses were established (4.8% absolute yield). The model was used as an explanation-first layer on top of existing genomic pipelines and did not make clinical decisions.
Key Points
- Dataset and result: 376 de-identified cases, 18 confirmed additional diagnoses (4.8% yield); cohort yields ranged from 1.0% to 13.3%.
- Inputs and workflow: each case packet included HPO phenotype terms, clinician notes/metadata, and a filtered variant table (rarity, predicted effect, ClinVar status, family signal). The model was prompted to propose a most plausible molecular explanation and show supporting evidence.
- Human-in-the-loop adjudication: model outputs were reviewed by at least two experts using ACMG/AMP criteria; final calls required expert consensus, CLIA confirmation, and return-to-family. The model never issued diagnoses or clinical decisions.
- Validation and heuristics: workflow recovered known diagnoses in internal validation sets (e.g., 48/51 and 45/57 in two cohorts); the model’s internal confidence correlated with correctness (mean min score ~85.6 for correct runs vs ~42.1 for incorrect), used as a reviewer triage heuristic, not a probability.
- Notable capabilities: the model inferred structural events (22q11.2 deletion), surfaced digenic explanations, and generated testable mechanistic hypotheses (e.g., S1PR1-vililigo link) that warrant experimental follow-up.
- Limitations and cautions: retrospective, heterogeneous cohorts; reviewers not blinded to confidence; operational challenges in aggregating distributed records and public database updates; explicitly not a clinical diagnostic tool.
Engineering implications / next steps
- Treat models as hypothesis generators integrated into reproducible, auditable pipelines that capture input packets, prompts, model outputs, and reviewer decisions.
- Prioritize data harmonization (IDs, vocabularies, ClinVar sync) to reduce missed rediscoveries and operational gaps.
- Use model confidence and validation runs to triage reviewer effort, but retain strict clinical adjudication (ACMG/AMP + CLIA) and logging for traceability.
- Plan prospective, blinded evaluations and automation-safe guardrails before any clinical deployment; periodic reanalysis pipelines could be implemented to scale expert review as knowledge evolves.