Literature
Display
← All papers

EchoAtlas: A Conversational, Multi-View Vision-Language Foundation Model for Echocardiography Interpretation and Clinical Reasoning

Chieh-Ju Chao · Mohammad Asadi · Lavonda Li · Gokul Ramasamy · Nicolo Pecco · Yu-Chiang Wang · Timothy Poterucha · Reza Arsanjani · Garvan Kane · Jae K Oh · Imon Banerjee · Curtis Langlotz · Li Fei-Fei · Ehsan Adeli · Bradley J. Erickson

Preprint posted · medRxiv (preprint) ↗

Healthcare AI12 min text · CondensedOriginal paper ↗
Contents

At a glance

EchoAtlas adapts Qwen2.5-VL-7B to conversational echocardiography using report-derived supervision paired with multiple ultrasound views. Tasks include questions, numerical estimates, regional wall motion, longitudinal comparison, structured reporting, and diagnostic explanations.

The dataset contains 12.88 million question–answer pairs from 62,726 patients. Internal multiple-choice accuracy is 0.966; external single-video accuracy is 435/622, or 0.699. These retrospective results support domain adaptation, while performance varies substantially by measurement, task format, and disease. This edition follows the complete 46-page medRxiv preprint posted 17 March 2026, including every figure, table and original prompt.

Complete original figure 1
Figure 1. Overview of EchoAtlas Study. Panel a. Development and Performance of EchoAtlas. EchoAtlas is an autoregressive vision-language model (VLM) trained on a large-scale dataset of 12.9 million question-answer pairs corresponding to ~2 million echocardiographic videos. Panel b. Benchmark performance. The model achieves 0.966 accuracy on internal MCQ test sets and establishes a new state-of-the-art (SOTA) on the public MIMIC-EchoQA benchmark (0.699 accuracy) compared to previous benchmarks (0.508). Panel c. In addition to MCQ, EchoAtlas also demonstrates robust performance across clinical tasks, including quantitative measurements, segment-level regional wall motion assessment, longitudinal study comparison, and complex diagnostic reasoning.

Introduction

Clinical echocardiography requires integrating anatomy, motion, Doppler information, and complementary views. Specialized predictors can estimate selected quantities, but conversational interpretation also requires flexible questions and descriptions [1] [2] [4] [11].

EchoAtlas uses an autoregressive model that conditions on videos and questions. This differs from EchoPrime’s retrieval-based reporting and EchoCLIP-style representation learning [5] [10]. The paper evaluates these approaches in specific formats; it does not establish a universal superiority of one architecture.

Results: conversational interpretation

Table 1 reports 0.966 accuracy on 63,752 multiple-choice questions and 0.943 on 34,146 closed-ended questions. Baseline multiple-choice accuracies are 0.161 for OctoMed, 0.422 for MedGemma and 0.363 for base Qwen. Domain training produces a large improvement under this internal protocol.

RadGraph F1 is 0.785 for description, 0.703 for open answers, 0.581 for comparison and 0.365 for reasoning. These entity/relation overlap scores are distinct from answer accuracy and do not directly establish clinical correctness or faithful reasoning. Supplementary Figures 1–5 retain the complete keyword-level results, including weaker categories.

The source’s sample counts need care: Table 1 gives 518 comparison, 558 reasoning and 1,801 template entries, whereas Supplementary Table 1 gives 33,554, 518 and 558 test entries respectively. The counting units or discrepancies are not resolved in the manuscript. Table 1 also states that 22 temporal-comparison ground-truth mappings were corrected before reporting results.

Complete original figure s1
Supplementary Figure 1. Model performance on multiple-choice questions stratified by clinical keyword. Heatmap displaying accuracy values (color scale 0–1; green indicates higher performance). EchoAtlas achieved above 0.9 accuracy across most categories, except for aortic stenosis (0.837) and left atrium (0.844), and outperformed all comparison models (OctoMed-7B, MedGemma-4B-IT, and Qwen2.5VL-7B) in every category.
Complete original figure s2
Supplementary Figure 2. Model performance on closed-ended questions stratified by clinical keyword. Heatmap displaying accuracy values (color scale 0–1; green indicates higher performance). EchoAtlas achieved above 0.9 accuracy across a majority of categories and outperformed all comparison models (OctoMed-7B, MedGemma-4B-IT, and Qwen2.5VL-7B) in every category, except for a tie in constrictive pericarditis.
Complete original figure s3
Supplementary Figure 3. Model performance on open-ended questions stratified by clinical keyword. Heatmap displaying RadGraph F1 values (color scale 0–1; green indicates higher performance). EchoAtlas achieved robust performance (>0.5) across a majority of categories and outperformed all comparison models (OctoMed-7B, MedGemma-4B-IT, and Qwen2.5VL-7B) in every category.
Complete original figure s4
Supplementary Figure 4. Model performance on Description questions stratified by clinical keyword. Heatmap displaying RadGraph F1 values (color scale 0–1; green indicates higher performance). EchoAtlas achieved robust performance (>0.5) across a majority of categories and outperformed all comparison models (OctoMed-7B, MedGemma-4B-IT, and Qwen2.5VL-7B) in every category.
Complete original figure s5
Supplementary Figure 5. Model performance on Comparison questions stratified by clinical keyword. Heatmap displaying RadGraph F1 values (color scale 0–1; green indicates higher performance). EchoAtlas achieved robust performance (>0.5) across a majority of categories, except for comparing aorta (0.462), aortic valve (0.330), left ventricle (0.492) and mitral valve (0.431). EchoAtlas outperformed all comparison models (OctoMed-7B, MedGemma-4B-IT, and Qwen2.5VL-7B) in every category. Note aortic valve values represent results after correcting the ground truth.
Complete original table 1
Table 1. Performance on the Mayo Clinic internal test set. Complete original table, including task-specific sample counts, confidence intervals and footnotes.

Results: numerical measurements and wall motion

Across 27,391 numerical predictions, the paper reports pooled R²=0.897 and mean absolute error 6.73. These aggregate heterogeneous quantities and units; the pooled error is not a single clinically interpretable measurement error. Per-parameter results in Figure 2 are more informative.

Examples include right-ventricular systolic pressure R²=0.845 with error 2.485, biplane left-ventricular end-diastolic volume R²=0.886 with error 10.94, and ejection fraction R²=0.773 from the two-chamber view versus 0.461 from the four-chamber view, with errors 2.54 and 4.68 percentage points. Performance is not uniformly strong: many strain and Doppler quantities have negative R². Spectral Doppler is absent from the input.

Methods and Figure 2 contain 38 numerical parameters, although Discussion says 39. Approximately 29% exceed R²=0.7 and 63% exceed 0.3. These are report-supervised estimates from videos, without a specialized measurement head or demonstrated direct caliper measurement. Joint-model blank cells indicate tasks not trained, rather than zero scores.

For 3,478 regional wall-motion assessments across 16 segments, the main text gives three-class accuracy 0.745, ranging from 0.586 to 0.892 by segment. Figure 2c instead shows 78.5% for EchoAtlas; the manuscript does not explain the aggregation difference. Binary normal/abnormal performance is higher, with reported positive and negative predictive values of 0.771 and 0.853. These tasks and numerical estimates have different metrics and denominators.

Complete original figure 2aComplete original figure 2bComplete original figure 2c
Figure 2a and 2b. Model performance for echocardiographic measurements assessed by R2 and MAE. Heatmaps showing R2 values (a) and MAE values (b) across multiple cardiac measurements predicted by different models. Higher R2 values and lower MAE values (green) indicate better prediction accuracy. The model achieved notably high R2 values for LV cardiac index (0.999), right ventricular systolic pressure (0.845), left atrial maximum volume by fourchamber view (0.913), and left ventricular end diastolic volume (0.886), with corresponding low MAEs of 0.02, 2.485, 5.80, and 10.94, respectively, demonstrating reliable quantitative estimation of key cardiac parameters. Ejection fraction predictions showed R2 values of 0.77 (two-chamber) and 0.46 (four-chamber) with MAEs of 2.54% and 4.68%, respectively. Figure 2c. Wall motion score classification performance across models. Heatmap comparing six vision-language models on 3,478 echocardiographic segments. Left panel: binary (normal vs. abnormal); right panel: 3-class (normal/hypokinetic/akinetic). Unparseable predictions were counted as failures across all metrics. EchoAtlas variants achieved binary accuracy of 0.833– 0.835, PPV of 0.771–0.786, and NPV of 0.845–0.853 for abnormality detection, representing 15–20 percentage point improvements over the base Qwen2.5VL-7B model. OctoMed-7B and MedGemma-4B-IT showed substantially lower performance (binary accuracy 0.560 and 0.542; PPV 0.230 and 0.354; NPV 0.680 and 0.690). Sen, sensitivity; Sp, specificity; PPV, positive predictive value; NPV, negative predictive value; Acc, overall accuracy.

Results: clinical reasoning

Reasoning answers are divided into observations, motion, hemodynamics, differential considerations, diagnostic criteria/integration, and summary. Automated scores are strongest for visual observation and criteria and weaker for motion and hemodynamics. Disease groups range from 140 aortic-stenosis cases to only four dissections and eight tamponades; rare-category results have limited precision.

Two cardiologists review 50 disease-stratified random cases. The main text reports relevance and coherence 5.00, completeness 4.96±0.14, and correctness 4.00±0.81. The Figure 3b caption instead gives completeness 4.94±0.25 and correctness 3.87±0.95. These differing summaries remain unresolved and are preserved in the original caption.

Agreement is reported as 86.4%, with κ=0.553. Readable, complete explanations can still contain incorrect diagnoses or plausible explanations of them. This small expert sample supports an assessment of answer quality, not proof that the generated reasoning faithfully reflects the model’s decision process.

Complete original figure 3aComplete original figure 3b
Figure 3a. Long-form reasoning performance across diagnostic subsections assessed by RadGraph F1 score. Heatmap showing model performance on different reasoning subsections (Diagnostic Criteria, Differential Diagnosis, Hemodynamics, Integration, Motion Pattern, and Visual Observations) across ten cardiac diagnoses. Higher RadGraph F1 scores (green, scale 00.5 for better visualization) indicate better alignment between model-generated and reference reports. EchoAtlas demonstrated robust long-form reasoning capabilities, with Visual Observations achieving the highest scores across all diagnoses (mean F1: 0.275-0.397), followed by Diagnostic Criteria (0.208-0.435) and Differential Diagnosis (0.123-0.326). Aortic Stenosis showed the strongest overall performance across subsections, while more complex diagnoses like hypertrophic cardiomyopathy and cardiac tamponade presented greater reasoning challenges. Figure 3b. Expert evaluation of reasoning quality across diagnostic categories. Heatmap showing model performance on different reasoning quality metrics (correctness, completeness, relevance, and coherence) across 8 diseases in a subset of randomly selected cases. Reasoning demonstrated near-perfect relevance (5.00±0.00) and coherence (5.00±0.00), strong completeness (4.94±0.25), and moderate correctness (3.87±0.95). Performance varied by disease category, with mitral regurgitation achieving the highest correctness (4.28) and cardiac tamponade the lowest (3.00).

Results: template reporting

Template-specific fine-tuning yields mean accuracy 0.783 (95% interval 0.763–0.803); joint training yields 0.787 (0.767–0.807). Their difference is small. Strong entries include mass/thrombus, tricuspid prosthesis and effusion; mitral-leaflet classification and fine-grained regurgitation grading remain substantially weaker. Supplementary Figure 6 retains all items.

Template numerical estimates deteriorate: joint/template-only R² is 0.294/0.137 for ejection fraction and 0.131/0.036 for right-ventricular systolic pressure. Template inputs use fewer clips per view, so this is not an isolated test of output formatting. Very low baseline results also reflect refusals or unparsable outputs.

Although Methods calls this a 29-question template, Supplementary Table 2 contains 28 questions: 26 categorical and two numerical, consistent with Supplementary Table 1. Every printed question and answer option is retained below.

Complete original figure s6
Supplementary Figure 6. Heatmap of Template Report performance across five models. The upper panel shows per-question accuracy (color scale 0–1) for 26 structured multiple-choice templates covering LV function (LV Fill. Pressure, LV Chamber Size, LV Wall Thickness, Regional WMA), right ventricular (RV) function (RV Chamber Size, RV Systolic Function), atrial anatomy (LA Size, RA Size, Atrial Shunt), valve morphology and disease severity (MV Leaflets, Mitral Annulus, MV Prosthesis, MR Severity, AV Stenosis, AV Morphology, AR Severity, TV Structure, TV Prosthesis, TR Severity, PR Severity), cardiac devices (Pacemaker/ICD Lead), vascular structures (IVC Size, Ascending Aorta Size), and other findings (Pericardial Effusion, Intracardiac Mass/Thrombus, Special Conditions). The lower panel shows R2 for two continuous measurements: left ventricular ejection fraction (LVEF R2) and right ventricular systolic pressure (RVSP R2), with gray cells indicating insufficient valid numeric predictions. Models are EchoAtlas-Temp, EchoAtlas-Joint, OctoMed-7B, MedGemma-4b-it, and Qwen2.5VL-7B. Abbreviations: WMA, wall motion abnormality; MV, mitral valve; MR, mitral regurgitation; AV, aortic valve; AR, aortic regurgitation; TV, tricuspid valve; TR, tricuspid regurgitation; PR, pulmonary regurgitation; IVC, inferior vena cava; ICD, implantable cardioverter-defibrillator; LVEF, left ventricular ejection fraction; RVSP, right ventricular systolic pressure.

Results: external validation

MIMIC-EchoQA provides 622 four-option questions, each paired with a single video from a different institution. EchoAtlas answers 435 correctly: accuracy 0.699 (95% interval 0.662–0.734). Figure 4 shows MedGemma 0.513, OctoMed 0.487, adapted EchoPrime 0.481 and base Qwen 0.219. The prior Echo-CoPilot value 0.508 is taken from the literature [18] [19].

This supports transfer on a retrospective single-clip benchmark. It does not evaluate complete-study reporting or prospective clinical outcomes. Template and joint variants score 0.665 and 0.333, showing that adding tasks does not automatically improve transfer. Performance also varies across structures and severity categories.

All 187 errors receive expert review. Averaged categories are 39.6% true model error, 39.3% ambiguous ground truth, 10.2% label noise, 5.6% view mismatch and 5.3% poor image quality. Reviewers agree on only 47.6% of categories, κ=0.25. These judgments identify benchmark ambiguity; they do not justify retrospectively increasing the reported accuracy.

Complete original figure 4aComplete original figure 4bComplete original figure 4c
Figure 4. EchoAtlas vs other models on MIMIC-EchoQA Benchmark. Panel a. Comparison of MCQ accuracy between EchoAtlas and other models. Error bars and bracketed values indicate the 95% CI for models. Echo-CoPilot (SOTA) is shown to represent the published accuracy from literature without CIs. All models were evaluated on the same test set (n=622 questions). EchoAtlas achieves 0.699 accuracy [95% CI: 0.662-0.734], significantly outperforming the current SOTA performance of Echo-CoPilot (50.8%) and other models. EchoPrime and Echo-CoPilot have similar performance to other general-purpose medical VLMs. Panels b and c. Model performance stratified by structure and view, respectively. EchoAtlas showed superior performance in nearly all categories except for the left atrium and right ventricle.

Discussion

The principal result is broad task coverage from large-scale echocardiography-specific supervision, with a clear internal advantage over the tested general models and measurable external transfer. Multiple views support integration, while format-specific and numerical weaknesses show that conversational fluency is insufficient.

Training labels originate in clinical reports and language-model transformations. They can carry omissions, incompatible views, or contradictory values. Low numerical performance without the required imaging modality is consistent with missing evidence, not necessarily a failure of language modeling alone. A favorable pooled score can conceal poor individual measurements.

The study is retrospective, largely from one health system, with 93.6% White participants in its reported cohort. External evaluation is narrower than the internal multitask suite. No prospective workflow study, patient-outcome benefit, or community-hospital generalization is demonstrated. Strong appearance and coherence of reasoning should therefore be considered separately from correctness.

The authors propose integrating additional clinical context, improving quantitative supervision and expanding reasoning and reporting. The current comparisons do not fully separate architecture, training data, image/video access, task mixtures and input budgets. Claims of inherent autoregressive advantages over retrieval require broader matched comparisons.

Methods: study cohort

The Mayo Clinic cohort comprises adult echocardiography studies from 2023 with research authorization. Supplementary Table 1 lists 12,879,662 question–answer pairs: 12,570,733 training, 102,467 validation and 206,462 test. The paper’s rounded 12.9 million total and 12.6 million training descriptions refer to different quantities.

Table 2 reports 1,942,439 videos, approximately 31 per patient, and 54.7% male participants. Dataset quantities should be read from their individual tables: the reported per-patient question average does not directly reconcile with the Supplementary Table 1 total.

Complete original table s1
Supplementary Table 1. Dataset size by question type and split. Complete original table, including task definitions and footnotes.

Methods: preprocessing and view classification

DICOM studies are divided into single/multiframe and Doppler categories. The model uses multiframe two-dimensional grayscale and color-Doppler clips, excluding spectral Doppler. Videos are cropped by 10%, resized to 224×224, and stored as MJPEG at 30 frames per second; non-imaging regions and the electrocardiogram trace are masked.

A DINOv2 ViT-B/14 view classifier maps 768-dimensional features through a 512-unit layer to 16 views [26]. Eleven uniformly sampled frames vote on each clip, with confidence at least 0.7. Uncertain labels and conflicting metadata are excluded. The authors report view-classification AUC near 0.99 and removal of 388,640 clips (19.1%); the filtering denominator is not fully reconciled with Table 2.

Methods: report-derived question generation

Cardiologists map report statements to appropriate views, excluding, for example, aortic-valve findings from an apical four-chamber view. GPT-4o then converts the mapped text into question–answer pairs. Crucially, it sees the reports, not the images, despite being prompted to assume it can observe them. Thus generated labels are report-derived rather than independently image-verified.

The licensed Mayo endpoint uses the reported API version 2024-10-21, maximum 6,000 tokens, temperature 0.1 and top-p 1.0. A low temperature does not by itself guarantee deterministic output. Eight formats cover general questions, measurements, comparisons, reasoning and templates.

The exact multiple-choice generation prompt is retained as the image labeled Supplementary Figure 8. Methods calls it Figure 9; the caption list assigns Figure 8 to the separate EchoPrime extraction prompt. This edition preserves both original prompt images and identifies the numbering conflict.

Complete original figure s8
Supplementary Figure 8 (label printed on the figure): original multiple-choice question-generation prompt. Editorial description; the supplied caption list does not contain a matching legend. The Methods instead refers to this prompt as Supplementary Figure 9.

Methods: multiview question tiers

General question types have three tiers: single-view structure, multi-view color Doppler, and multi-view structure with diagnostic integration. Mean clip counts are 2.3, 2.7 and 5.7 respectively. The tiers organize supervision and input complexity; they are not independent randomized tests of the benefit of adding views.

Methods: patient-level split

Splits are at patient level: 61,260 training patients, 493 validation and 973 test, or approximately 97.7%, 0.8% and 1.6%. This separates patients across partitions while allowing many correlated videos and questions per patient within each partition. Table 2 retains the complete demographic and data counts.

Complete original table 2
Table 2. Patient characteristics, video counts and question–answer counts by patient-level split. Complete original table.

Methods: measurement, comparison, reasoning and template tasks

Measurement supervision includes 38 numerical parameters and 16 categorical wall-motion segments. Longitudinal questions compare the same type of finding between studies from patients with at least two studies during the year. Reasoning answers follow six prescribed subsections.

Template questions group clinical fields by the most relevant views. Their answers are generated from the source report by the same language model. To fit memory limits, template inputs use one or two clips per view, compared with three to five for other multiview tasks. Both pages of the exact 28-question table are retained.

Complete original table s2 part1Complete original table s2 part2
Supplementary Table 2. Template Questions. Both original pages retained; the tricuspid-prosthesis question continues onto the second page. The printed table contains 28 questions.

Methods: model selection

The authors select Qwen2.5-VL-7B using existing medical-image and video benchmark performance relative to Qwen2-VL and InternVL 3.5 [27] [28] [29]. LoRA reduces adaptation cost [30]; training uses LLaMA-Factory. This selection rationale is not a controlled architecture ablation on the full EchoAtlas dataset.

Methods: staged training and settings

EchoAtlas trains on all question types except template reporting. EchoAtlas-Temp adds one template epoch to that checkpoint. EchoAtlas-Joint trains on the combined mixture from initialization, using a subset described as the 13 best-performing numerical tasks. Differences therefore include task mixture, numerical-task selection and training exposure.

Settings are LoRA rank 16, batch two per device, eight gradient-accumulation steps, learning rate 10⁻⁴, cosine decay and warm-up ratio 0.1. The paper reports one epoch for the base model, two for Temp, and approximately 190 hours on four 80 GB A100 GPUs, or 760 GPU-hours.

Methods: comparison models

Baselines are base Qwen2.5-VL-7B, OctoMed-7B and MedGemma-IT-4B [13] [14]. MedGemma receives a single frame per clip because its tested interface supports images rather than videos. Input modality and domain specialization consequently differ across models; this is not a parameter-matched architecture-only comparison.

Methods: expert reasoning review

Two independent cardiology experts rate 50 disease-stratified random cases on five-point correctness, completeness, relevance and coherence scales, following earlier evaluation work [23] [31] [32]. Table S3 retains their definitions. Correctness targets false statements; completeness targets missing findings. These dimensions should not be collapsed into a single claim of reliable diagnosis.

Complete original table s3
Supplementary Table 3: Reasoning Quality Metrics. Complete original definitions of correctness, completeness, relevance and coherence.

Methods: external benchmark

The external benchmark derives from Beth Israel Deaconess studies collected in 2012–2019. It contains 622 single-video, four-option questions generated from reports, filtered for view appropriateness and checked by cardiologists [17] [19]. Site, acquisition period and question format change together, so the experiment does not isolate temporal robustness.

Methods: EchoPrime adaptation

EchoPrime normally combines a full study’s clips for retrieval-based reporting. Here it is adapted to a single clip; predicted metrics and retrieved text are sent to the reported GPT-5 endpoint to extract the multiple-choice answer. The paper acknowledges that this is an out-of-distribution use of EchoPrime.

Supplementary Figure 7 shows conflicting outputs: predicted ejection fraction about 45.87%, while retrieved text describes 55–56% and normal function. Prioritizing the text yields roughly 0.44 accuracy; prioritizing predicted metrics yields roughly 0.48, the configuration used in the main comparison. This example exposes a conflict in the adapted pipeline, not a general impossibility of retrieval-based reasoning.

The exact extraction prompt is retained as Figure S9, its printed label. The accompanying caption calls it Figure 8. It instructs the model to use only supplied evidence and deductions, with direct metric predictions taking priority.

Complete original figure s7
Supplementary Figure 7. EchoPrime evaluation. During evaluations, we noticed that EchoPrime often generates contradictory values in the predicted metrics and the retrieved text, as shown in the figure. The metric predicted for ejection fraction is 45.87%, which would be categorized as mild LV systolic dysfunction, whereas the text reports it as 55-56%, as well as explicitly suggesting normal left ventricular systolic function. We found ~0.44 accuracy when prioritizing the report and 0.48 with the metrics taking priority. The latter is reported in the main text.
Complete original figure s9
Supplementary Figure 8. The prompt used for evaluating EchoPrime using GPT-5 (reasoning_effort=high). GPT-5 is instructed to extract the answer to the question only based on the provided (and the natural deductions from) information generated by EchoPrime. Metric values directly predicted get the priority. [Source numbering note: the preceding original legend says Figure 8; the prompt image itself is labeled Figure S9.]

Methods: external error review

Two cardiologists independently classify every incorrect external answer using five categories: model error, ambiguous ground truth, label noise, view mismatch and poor image quality. Table S4 retains the definitions and examples. Their averaged category distribution and fair agreement describe the audit; they do not replace the original benchmark labels.

Complete original table s4
Supplementary Table 4: Error Categories for Model Performance Analysis. All five original categories, definitions and examples.

Methods: statistical analysis

Accuracy evaluates categorical answers, RadGraph F1 evaluates clinical entities and relationships in free text, and R²/mean absolute error evaluate numerical estimates [34]. Wall-motion reporting additionally includes sensitivity, specificity and predictive values for three-class and binary tasks. These measurements answer different questions.

The paper uses 1,000 bootstrap replicates for intervals and 1,000 paired permutations for model comparisons, with a 0.05 threshold. It does not state patient-cluster resampling or multiple-comparison adjustment. Repeated questions from the same patient may therefore affect uncertainty estimates.

Echo-CoPilot’s external result is literature-derived, and Figure 4 does not show its confidence interval. The manuscript’s paired-test description should not be assumed to establish paired significance against that prior result without access to the corresponding predictions.

Ethical review and approval

Mayo institutional review approved protocol 22-010944. Included patients had authorized minimal-risk retrospective research; additional consent was waived. The study states compliance with the Declaration of Helsinki.

Data availability

Mayo data remain restricted for privacy; access is described as requiring a reasonable request and institutional review approval. MIMIC-EchoQA is available through PhysioNet. The manuscript does not explicitly establish an unrestricted release of model weights, code or the private training cohort.

Acknowledgments

Chieh-Ju Chao reports NIH/NHLBI K08 HL175205-01A1 and Mayo Clinic awards. Mohammad Asadi reports Amazon AI and Stanford HAI fellowships. Other authors declare no relevant funding for this work.

Author contributions

Chao leads conceptualization, methods, software, validation, curation, analysis and writing; Asadi contributes methods, software and writing. Li contributes validation, curation, writing and visualization; Ramasamy, Pecco, Wang and Poterucha support data preparation. Arsanjani, Kane, Oh, Banerjee, Langlotz, Li, Adeli and Erickson contribute resources, methodology, review and/or supervision as listed in the source. All authors approved the manuscript.

Concept demonstration

Open image ↗