ImageCLEFmed MEDIQA-CORE

Welcome to the first edition of the MEDIQA-CORE challenge!

Motivation

We propose MEDIQA-CORE: Multimodal Reasoning & Reconciliation in Radiology, a new challenge designed to advance clinical AI systems capable of complex medical understanding and decision support. This competition comprises two complementary tasks that evaluate model performance on diagnosis accuracy and clinical report alignment using multimodal data. 

Preliminary Schedule

• 28.01.2026: Registration opens
• 21.03.2026: Registration closes
• 27.03.2026: Release of the development set
• 23.04.2026: Release of the test sets
• 07.05.2026: Run submission deadline
• 12.05.2026: Release of the processed results by the task organizers
• 28.05.2026: Submission of participant papers [CEUR-WS]
• 30.06.2026: Notification of acceptance
• 06.07.2026: Camera ready copy of participant papers and extended lab overviews [CEUR-WS]
• 21-24.09.2026: CLEF 2026 conference, Friedrich-Schiller-Universität Jena, Germany

All times are in Anywhere on Earth (AoE) timezone

Task 1: Brain Tumor Subtype Classification

This task focuses on predicting tumor type labels by jointly analyzing brain MRI scans (including T1, T1c, T2, and FLAIR), H&E-stained pathology slides, and pathology reports. The objective is to develop models that can integrate visual and textual information from radiology and pathology to arrive at an accurate tumor classification. This is a challenging problem because it requires reasoning across multiple types of data that differ in resolution, format, and clinical context. A successful solution could improve diagnostic accuracy and reduce delays by enabling earlier decision-making while molecular biomarker results are still pending. 

This challenge focuses on automated brain tumor subtype classification using multimodal clinical data. Participants are provided with:

• Preoperative brain MRI sequences (T1, T1c, T2, FLAIR)
• H&E-stained whole-slide pathology images (when available)
• Corresponding free-text pathology reports (when available)

The objective is to develop machine learning models that assign each case to one predefined clinical tumor subtype by integrating radiologic, histologic, and textual information.

To reflect real-world clinical workflows, not all cases contain complete multimodal data. For a subset of patients, pathology data may be entirely unavailable at inference time. Cases may therefore include:1) MRI + Histopathology + Report or 2) MRI only.

All modality availability will be explicitly indicated. No case may be excluded due to missing data.

Participants must design models that:

• Utilize all available modalities when present
• Generate predictions when pathology data is missing
• Maintain robustness under incomplete input conditions

This design mirrors clinical practice, where imaging is immediately available, while histopathologic and molecular confirmation may be delayed or unavailable at early decision points.

Evaluation Platform & Leaderboard:
https://ai4media-bench.aimultimedialab.ro/competitions/6/


Task 2: Radiology Report Discrepancy Assessment


This task focuses on multimodal radiology report review and discrepancy analysis. Given a 3D medical image, a preliminary radiology report, and proposed candidate edits, the objective is to evaluate discrepancies between report versions and assess their clinical significance. The task reflects real-world clinical workflows in which junior radiologists draft reports that are subsequently reviewed and revised by attending radiologists.


Participants are provided with:

• 3D abdominal CT examinations
• Preliminary radiology reports
• Candidate report edits

The goal is to develop models that evaluate each proposed edit by:

• Determining image-level agreement between the edit and the CT findings
• Assessing the clinical severity of the discrepancy
• Classifying the discrepancy type (correction, addition, or clarification)

This task emphasizes fine-grained clinical reasoning and image–text alignment during the report review stage. Models must reason jointly over volumetric imaging data and structured clinical language to determine whether a proposed modification improves accuracy, introduces error, or refines interpretation.

The dataset consists of expert-annotated abdominal CT studies curated to reflect authentic report revision workflows. Standardized evaluation protocols will be provided to enable systematic comparison of multimodal models in terms of discrepancy detection accuracy, severity assessment, and edit classification performance.

This task promotes the development of clinically grounded multimodal systems capable of functioning as intelligent reviewers in radiology reporting pipelines.

Evaluation Platform & Leaderboard:
https://ai4media-bench.aimultimedialab.ro/competitions/7/


Data


The competition will utilize real, de-identified clinical images and reports collected from multiple U.S. hospitals. Together, these datasets and tasks reflect real-world challenges in clinical practice, where integrating diverse information sources and reconciling inconsistent reports are essential for safe and effective care.

Participant registration


Please refer to the general ImageCLEF registration instructions

Note: Registration for MEDIQA-CORE may take a few days to be accepted, as the forms have to be verified.

Task 1 Results


MEDIQA-CORE-Task 1:
https://docs.google.com/spreadsheets/d/1mNvA-01T8rtZYxsfubg_peqa6ClK9WnQnidvE5VtCsA/edit?usp=sharing




Task 2 Results


MEDIQA-CORE-Task 2:
https://docs.google.com/spreadsheets/d/1nIVBxMWllnybBtpK96C-B-70SOHQL9V-tdBj_C84E_M/edit?usp=sharing




CEUR Working Notes


Paper Submission Instructions

The full schedule is available at:
https://clef2026.clef-initiative.eu/dates/


Important Dates

• End of evaluation cycle (submission of runs): 7 May 2026
• Submission of participant papers (CEUR-WS): 28 May 2026
• Notification of acceptance for participant papers (CEUR-WS): 30 June 2026
• Camera-ready submission of participant papers: 6 July 2026

All submissions, reviews, and camera-ready versions will be handled through EasyChair: EasyChair CLEF 2026
A separate EasyChair track will be created for each lab/workshop. Please make sure you submit in the correct track.
The papers will go through a review process and will receive a decision from the lab organizers.

The participant papers should be written using the template provided here:
CLEF 2026 Working Notes Submission Template


Submissions are expected to be in English language and 5 pages minimum, with no maximum page limit.

Citations (Task 1)


When referring to MEDIQA-CORE-Task 1 competition and official results, please cite the following paper:
@inproceedings{mediqa-core-task-1,
author = {Asma {Ben Abacha} and
Juampablo E. {Heras Rivera} and
Daniel K. Low and
Wen{-}wai Yim and
Jacob Ruzevick and
Dan Child and
Mehmet Kurt},
title = {Overview of the MEDIQA-CORE 2026 Task 1 on Brain Tumor Subtype Classification},
booktitle = {CLEF 2026 Working Notes},
series = {CEUR Workshop Proceedings},
publisher = {CEUR-WS.org},
address = {Jena, Germany},
month = {September},
year = {2026}
}

When referring to MEDIQA-CORE-Task 1 dataset and baseline system, please cite the following paper:
@inproceedings{CoRe-BT-arXiv,
author = {Juampablo E. {Heras Rivera} and
Daniel K. Low and
Xavier Xiong and
Jacob J. Ruzevick and
Daniel D. Child and
Wen-wai Yim and
Mehmet Kurt and
Asma {Ben Abacha}},
title = {CoRe-BT: A Multimodal Radiology-Pathology-Text Benchmark for Robust Brain Tumor Typing},
journal = {CoRR},
volume = {abs/2603.03618},
year = {2026},
url = {https://arxiv.org/abs/2603.03618}
}

Citations (Task 2)


When referring to MEDIQA-CORE-Task 2 competition and official results, please cite the following paper:
@inproceedings{mediqa-core-task-2,
author = {Asma {Ben Abacha} and
Zhaoyi Sun and
Wen{-}wai Yim and
Fei Xia and
Meliha Yetisgen},
title = {Overview of the MEDIQA-CORE 2026 Task 2 on Radiology Report Discrepancy Assessment},
booktitle = {CLEF 2026 Working Notes},
series = {CEUR Workshop Proceedings},
publisher = {CEUR-WS.org},
address = {Jena, Germany},
month = {September},
year = {2026}
}

When referring to MEDIQA-CORE-Task 2 dataset and baseline system, please cite the following paper:
@article{sun2026radar,
title = {RADAR: A Multimodal Benchmark for 3D Image-Based Radiology Report Review},
author = {Sun, Zhaoyi and
Jagtiani, Minal and
Yim, Wen{-}wai and
Xia, Fei and
Gunn, Martin and
Yetisgen, Meliha and
Ben Abacha, Asma},
journal = {CoRR},
volume = {abs/2603.06681},
year = {2026},
url = {https://arxiv.org/abs/2603.06681}
}

Citations (ImageCLEF)


When referring to ImageCLEF 2026, please cite the following paper:

@inproceedings{ImageCLEF2026,
title = {Overview of ImageCLEF 2026: Multimodal Challenges in Medicine, Science, Agritech, and Security},
author = {Bogdan Ionescu and Henning M{\"u}ller and Dan{-}Cristian Stanciu and Andrei Radu and Radu{-}George Bolborici and Marian Negru and Alexandru{-}Florin Ene and Vlad{-}Mihai Vasilescu and Ana-Antonia Nicolae and Liviu{-}Daniel \c{S}tefan and Mihai{-}Gabriel Constantin and Mihai Dogariu and Alexandra{-}Georgiana Andrei and Hendrik Damm and Tabea M. G. Pakull and Asma {Ben Abacha} and Alba {Garc\'ia Seco de Herrera} and Christoph M. Friedrich and Raphael Br{\"u}ngel and Lea Reinartz and Henning Sch{\"a}fer and Cynthia Sabrina Schmidt and Benjamin Bracke and Praveen Nath and Bahad{\i}r Ery{\i}lmaz and Maja Hjuler and Diandra Fabre and Claire Lemaire and Benjamin Lecouteux and Didier Schwab and Dimitar Dimitrov and Ming Shan Hee and Momina Ahsan and Sarfraz Ahmad and Dimitrina Zlatkova and Georgi Pachov and Zhuohan Xie and Preslav Nakov and Ivan Koychev and Juampablo E. {Heras Rivera} and Daniel K. Low and Wen{-}wai Yim and Jacob Ruzevick and Dan Child and Mehmet Kurt and Zhaoyi Sun and Fei Xia and Meliha Yetisgen and Ahmedkhan Radzhabov and Yuri Prokopchuk and Vassili Kovalev and Dzmitry Karpenka and Steven A. Hicks and Sushant Gautam and Michael A. Riegler and Vajira Thambawita and P\r{a}l Halvorsen and Mohammad {El Sakka} and Josiane Mothe and Alexandra B\u{a}icoianu and Corneliu{-}Nicolae Florea and Mihai Ivanovici},
booktitle = {Experimental IR Meets Multilinguality, Multimodality, and Interaction},
series = {Proceedings of the Seventeenth International Conference of the CLEF Association (CLEF 2026)},
year = {2026},
month = {September 21--24},
address = {Jena, Germany},
publisher = {Springer Lecture Notes in Computer Science LNCS},
}

Task 1 Organizers

• Asma Ben Abacha, Microsoft
• Juampablo E. Heras Rivera, University of Washington
• Daniel K Low, University of Washington
• Wen-wai Yim, Microsoft
• Dr. Jacob Ruzevick, University of Washington
• Dr. Dan Child, University of Washington
• Mehmet Kurt, University of Washington

Task 2 Organizers

• Asma Ben Abacha, Microsoft
• Zhaoyi Sun, University of Washington
• Wen-wai Yim, Microsoft
• Fei Xia, University of Washington
• Meliha Yetisgen, University of Washington

Contact Information

• MEDIQA Mailing List: https://groups.google.com/g/mediqa-nlp
• If you have any questions, please email us at mediqa.organizers@gmail.com