Gorio Tech Blog search

MedGemma Technical Report | Summary

한국어로 읽기

|
Contents

This article reviews MedGemma Technical Report by Google Research and Google DeepMind.

Image attribution. Source: Google Research and Google DeepMind, MedGemma Technical Report, arXiv:2507.05201v4, © 2026 Google. All rights reserved. The arXiv record lists the paper under CC BY 4.0. The images are cropped from the source PDF: Figure 1 (p. 2), and Tables 4, 7 and 13 (pp. 9, 14 and 17).

Source version and provenance. This is a manual retrospective review of arXiv v4, dated 6 April 2026; the report itself is dated 8 April 2026. The results below belong to that later revision and should not be read as measurements available in July 2025.

Read this article in Korean


Summary

  • MedGemma adapts Gemma 3 for medical text and images. The main evaluation covers a 4B multimodal model and a 27B text-only model; a separate 27B multimodal variant has preliminary results in Appendix F. Both main models produce text.
  • On the out-of-distribution CheXpert chest X-ray benchmark, zero-shot MedGemma 4B reaches macro F1 48.1 versus 32.6 for Gemma 3 4B. On text-only MedXpertQA, the 27B models score 25.7% and 15.7% accuracy, respectively. These are benchmark comparisons under the report’s evaluation conditions, not estimates of clinical performance.
  • Further adaptation improves selected tasks, including a synthetic electronic health record question-answering benchmark. The authors describe benchmark evaluation as an initial step; the report does not establish safety or effectiveness in a clinical workflow.

1 Introduction

Healthcare data spans medical images and text, while task-specific models often require substantial labeled data. MedGemma is intended as a foundation for developers who want to adapt an open medical model. The collection also includes MedSigLIP, a medical image encoder. Figure 1 on page 2 summarizes the 4B multimodal model, the 27B text model, and the encoder.

The overview connects four medical imaging domains and medical text to the model collection. MedSigLIP provides the image encoder for the 4B multimodal model; the 27B model shown here is text-only.

MedSigLIP, MedGemma 4B multimodal, and MedGemma 27B text shown with medical image and text inputs.
Overview of the MedGemma collection and its medical image and text inputs.

The 4B model accepts images, text, or both. In the main report, “MedGemma 27B” denotes the text-only variant unless stated otherwise. The 27B multimodal model is distinct, and its evaluation in Appendix F is preliminary.

2 Methods

2.1 Data and preprocessing

The training data combines general-purpose data with medical text and images. Medical text post-training includes teacher responses and logits for several question-answering datasets and about 200,000 synthetic questions. The multimodal data includes radiology, histopathology, dermatology, ophthalmology, and medical literature images. Table 1 on page 4 lists datasets and training stages.

The image scope is two-dimensional: X-rays and selected 2D CT or MRI slices are included, while 3D volumes and genomic data are excluded. Training images are resized to 896 × 896 pixels. CT slices use three window settings mapped to RGB channels to highlight bone and lung, soft tissue, and brain structures.

2.2 Architecture and training

MedGemma follows Gemma 3 and uses its 400M-parameter SigLIP image encoder. The 4B model’s encoder is enhanced with more than 33 million medical image-text pairs, predominantly histopathology patches, while retaining general data; medical examples have a 2% mixing weight during this stage. The released standalone MedSigLIP uses the same weights at 448 × 448 resolution, with downsampled positional embeddings.

The 4B multimodal decoder is then pretrained with medical image-text data at a 10% mixing weight, alongside the original general mixture. Medical text is added during distillation, and paired medical images and text are used in reinforcement-learning post-training. The text-only 27B variant uses the post-training stage rather than the full multimodal sequence. These details are described on pages 4–5.

3 Evaluation design

The report evaluates medical text questions, image classification, visual question answering, chest X-ray report generation, and simulated agent behavior, as well as general benchmarks. Table 2 on page 6 identifies datasets and out-of-distribution (OOD) tasks, meaning data not used during model development. Unless otherwise stated, the authors use one inference per example and temperature 0 for MedGemma on medical benchmarks. Comparison models may use different default sampling settings.

The radiology evaluation uses different labels and conditions across datasets. The MIMIC-CXR test variants differ in their treatment of missing and uncertain labels; CheXpert and ChestX-ray14 evaluate different condition sets. Some dermatology and pathology evaluations use internal datasets, so their results cannot be treated as broad clinical validation.

For report generation, a single board-certified thoracic radiologist reviewed 306 cases. The reviewer was not blinded to the origin of the reports. AgentClinic uses simulated clinical encounters, with a model acting as a patient and evaluator; it is not a prospective patient-care study.

4 Results

4.1 Medical knowledge and image understanding

On the OOD MedXpertQA text subset, Table 4 on page 9 reports accuracy of 14.2% for MedGemma 4B versus 11.6% for Gemma 3 4B, and 25.7% for MedGemma 27B versus 15.7% for Gemma 3 27B. On the multimodal-only subset, the 4B scores are 24.4% and 22.3%. The corresponding differences are 2.6, 10.0, and 2.1 percentage points, respectively.

The text-only and multimodal-only rows compare distinct MedXpertQA subsets. The text-only MedGemma 27B cannot be evaluated on the multimodal-only row.

MedXpertQA accuracy for text-only and multimodal-only subsets across MedGemma, Gemma 3 and larger models.
Accuracy on out-of-distribution MedXpertQA subsets.

Table 7 on page 14 reports zero-shot generative chest X-ray classification. MedGemma 4B and Gemma 3 4B obtain macro F1 scores of 48.1 and 32.6 on CheXpert, and 50.1 and 32.0 on ChestX-ray14. CheXpert averages five conditions, whereas ChestX-ray14 averages three; the latter uses radiologist-adjudicated labels. The 15.5- and 18.1-point differences should not be interpreted as relative percentage increases or as direct evidence of patient benefit.

MIMIC-CXR and CheXpert average five findings, while CXR14 averages lung opacity, pneumothorax, and fracture. The starred rows use radiologist-adjudicated labels, and the RadVLM comparison is estimated from a published bar chart.

Chest X-ray zero-shot macro F1 by dataset, model, and evaluated conditions, with label and comparison footnotes.
Zero-shot chest X-ray classification, with dataset-specific findings and label qualifications.

For open-ended visual questions, the authors report results on SLAKE and VQA-RAD. They use a VQA-RAD split designed to avoid duplicate train-test images, which limits direct comparison with results using the original split. Figures 2 and 3 illustrate individual model responses rather than aggregate performance.

Table 9 on page 15 reports overall token F1 of 72.3 versus 40.2 for MedGemma 4B and Gemma 3 4B on English SLAKE, and 49.9 versus 33.6 on the authors’ VQA-RAD split. Table 8 on page 14 reports accuracy on internal image classification tasks formulated as zero-shot multiple-choice questions: MedGemma 4B scores 69.8 on pathology, 71.8 on dermatology, and 64.9 on retinal images. These results depend on those datasets and question formats.

4.2 Reports and simulated agents

For chest X-ray report generation, the authors use a pretrained checkpoint because the metric is sensitive to reporting style. MIMIC-CXR reports were part of training, and the authors say that the pretrained checkpoint better follows their style than the post-trained model; this is a relevant influence on the comparison, not proof of overlapping test images. In the 306-case expert review, 81% of generated reports were judged to lead to the same or better clinical decisions than the original reports. That judgment comes from one unblinded reviewer and is specific to this sample and evaluation design (pages 8 and 13, Figure 4).

On the OOD AgentClinic-MIMIC simulation, Table 11 on page 16 reports 46.0% accuracy for MedGemma 27B and 35.2% for Gemma 3 27B, a 10.8-point difference. The table’s historical human-physician number uses a GPT-4 patient agent, whereas the recalculated model evaluations use GPT-4o. It is therefore not a controlled head-to-head comparison with clinicians.

On the general, text-only MMLU Pro benchmark, Table 12 on page 16 reports 39.1% for MedGemma 4B versus 43.6% for Gemma 3 4B, and 60.2% versus 67.5% for the 27B models. Medical adaptation therefore accompanies lower MMLU Pro accuracy in these comparisons. The authors also caution on page 14 that the public availability and frequent use of established benchmarks raise the possibility of test-data leakage; they do not establish that specific test examples leaked into these models’ training data.

5 Task-specific adaptation

The authors fine-tune MedGemma 4B with supervised learning for image tasks. Table 13 on page 17 reports SIIM-ACR pneumothorax accuracy rising from 85.9 to 87.8 and F1 from 59.7 to 71.5. On CRC100k histopathology patch classification, weighted F1 rises from 32.8 to 94.5. These are results for the stated datasets and fine-tuning setup, not out-of-box performance.

The same table reports MIMIC-CXR report-generation RadGraph F1 of 29.5 for the pretrained checkpoint and 30.3 for a model fine-tuned from the instruction-tuned MedGemma 4B checkpoint. These scores do not compare the same checkpoint before and after fine-tuning. This metric compares generated text with reference reports and remains sensitive to reporting style.

The table compares out-of-box and adapted results for three tasks. For MIMIC-CXR report generation, 29.5 uses the pretrained checkpoint and 30.3 uses a model fine-tuned from an instruction-tuned checkpoint.

MedGemma 4B fine-tuning results for chest X-ray reports, pneumothorax classification and histopathology.
Task-specific adaptation results, including report-generation scores from different starting checkpoints.

For electronic health record retrieval and reasoning, the authors apply reinforcement learning to MedGemma 27B. Table 14 on page 18 reports EHRQA accuracy increasing from 86.3% to 93.6%. The benchmark uses synthetic FHIR records: 81 records fit the context limit and were split into 42 training, 20 validation, and 19 test records. The records lack clinical notes. The comparison does not measure performance on real patient records or establish clinical reliability.

6 MedSigLIP and the preliminary multimodal 27B model

MedSigLIP is evaluated as an image encoder using zero-shot classification and linear probes, which train a simple classifier on frozen representations. Tables 15 and 16 on page 20 report AUC across domains and chest X-ray findings. Results vary by condition and comparator; an average across selected findings should not be read as a universal clinical accuracy.

Figure 5 on page 19 plots average linear-probe AUC against training sample size for seven chest X-ray findings across CheXpert and ChestX-ray14. The authors describe MedSigLIP as performing strongly when at least 512 training examples are available; the curve concerns this evaluation and should not be generalized to every image task.

Appendix F on page 60 describes the separate 27B multimodal variant. Its training adds EHRQA and Chest ImaGenome data to the multimodal recipe. Table A14 is explicitly preliminary: for example, its CheXpert score is 49.9 versus 48.1 for 4B, while its ChestX-ray14 score is 45.3 versus 50.1. Neither variant wins every listed task.

7 Limitations and interpretation

Benchmarks provide a snapshot of model behavior, not evidence that a complete application is safe or effective in practice. Dataset composition, label conventions, limited expert review, simulated agent settings, and synthetic records constrain the conclusions. The report calls for further evaluation on challenging benchmarks and real application requirements. Any downstream medical use requires validation in its intended population and workflow.