ImageCLEF 2025

Motivation

Given the successful ImageCLEF 2003-2024 benchmarking campaigns, ImageCLEF 2025 edition (https://www.imageclef.org/, https://www.facebook.com/imageclef, https://twitter.com/imageclef) will continue to evaluate the technologies for annotation, indexing, classification and retrieval of multimodal data, with its main objective residing in providing access to large collections of multimodal data for multiple usage scenarios and domains.
Considering the experience of the last editions, ImageCLEF 2025 will continue approaching a diversity of applications, namely medical, social media and Internet, and recommending, giving to the participants the opportunity to deal with interdisciplinary approaches and domains. ImageCLEF 2025 will continue the experience of the engaging tasks from 2024 edition, while some of the other tasks are discontinued.
ImageCLEF 2025 is organized as part of the CLEF Initiative Labs.

The results of the campaign appear in the working notes proceedings, published by CEUR Workshop Proceedings (ceur-ws.org, e.g., 2024 https://ceur-ws.org/Vol-3740/). Selected contributions among the participants, will be invited for publication in the following year in the Springer Lecture Notes in Computer Science (LNCS) together with the annual lab overviews (e.g., 2024 https://link.springer.com/book/10.1007/978-3-031-71908-0).

The target audience for ImageCLEF 2025 is mainly expected to be from the multimodal data annotation and retrieval community, from fields such as computer vision, image information retrieval and digital image processing. Due to the success and the specific nature of the medical tasks, a significant part of the audience will come from the medical informatics, machine learning and pattern recognition community.

Stay tuned with us for the latest information and updates by joining us on the ImageCLEF social media accounts: Twitter #imageclef, and Facebook @ImageClef.

ImageCLEF2025 schedule

Each of the tasks sets its own schedule, so please check the corresponding task webpage for specific dates. A (tentative) global schedule can be found below:

• 20.12.2025: registration opens for all ImageCLEF tasks
• 25.04.2025: registration closes for all ImageCLEF tasks
• 24.01.2025: development data release starts (depends on the task)
• 14.03.2025: test data release starts (depends on the task)
• 10.05.2025 : deadline for submitting the participants runs (depends on the task)
• 17.05.2025: release of the processed results by the task organizers (depends on the task)
• 30.05.2025: deadline for submission of working notes papers by the participants
• 27.06.2025: notification of acceptance of the working notes papers
• 07.07.2025: camera ready working notes papers
• 09-12.09.2025: CLEF 2025, Madrid, Spain

The CLEF Conference

ImageCLEF lab and all its tasks are part of the Conference and Labs of the Evaluation Forum: CLEF 2025. CLEF 2025 will be hosted by the UNED University at Madrid, Spain, 9-12 September 2025 and consists of an independent peer-reviewed workshops on a broad range of challenges in the fields of multilingual and multimodal information access evaluation, and a set of benchmarking activities carried in various labs designed to test different aspects of mono and cross-language information retrieval systems. More details about the conference can be found here. Also there is more information about the CLEF Initiative.

Participant registration

ImageCLEF is an evaluation lab and starting from this edition it will be managed by the ImageCLEF team through the AI4Media benchmarking platform (based on Codalab). The main features of the platform are: online registration system, end-user agreement submission, data distribution, a leaderboard system as well as interaction with participants, reducing the administrative overhead. The platform is now up for participants' registration.

https://ai4media-bench.aimultimedialab.ro/competitions/public/

Please note that the registration is a two-step process. In order to participate in an ImageCLEF task, you must first accept its specific terms and conditions, which are listed on the 'Get Started -> Terms' tab of the tasks hosted on the AI4MediaBench platform. The registration form is mandatory. Within the form, you have the option to choose one or more ImageCLEF tasks from the provided list. Once the form is completed, your application will be applicable to all selected tasks hosted on AI4MediaBench. After completing the form, you must apply to the tasks you have selected in the terms and conditions via the 'Participate' tab of the tasks.

The Tasks

ImageCLEF 2025 proposes 4 main tasks:

(9th edition) ImageCLEFmedical: Multimodal data can be used in different scenarios. For example, manual generation of the knowledge of medical images is a time-consuming process prone to human error. As this process requires assistance for the better and easier diagnoses of diseases that are susceptible to radiology screening, it is important that we better understand and refine automatic systems that aid in the broad task of radiology-image metadata generation. Thus, we proposed:

• (9th edition) Automatic Image Captioning;
• (3rd edition) Synthetic Medical Images Created via GANs;
• (3rd edition) Visual Question Answering;
• (3rd edition) Multimodal And Generative TelemedICine (MAGIC).

(4th edition) Image Retrieval/Generation for Arguments: Images can strengthen an argumentative position through visual communication, but traditional image search is not well suited for finding images that support a particular point of view. This task aims to close this gap. This year we streamlined the task further and explored the task for concrete claims.

(2nd edition) ImageCLEFtoPicto: This task is designed for individuals with language impairments who use pictograms as a communication aid. The main usage scenario involves converting either text or speech into a meaningful sequence of pictograms, facilitating communication between verbal individuals and AAC users.

(1st edition) ImageCLEF MultimodalReasoning: Vision-Language Models (VLMs) excel in tasks combining vision and language, like image captioning and basic visual question answering. However, they often falter in deep logical reasoning and handling complex dependencies or hypothetical scenarios. This task aims to evaluate modern LLMs' reasoning abilities on intricate, multilingual inputs across diverse subjects.

Overview Paper

@inproceedings{OverviewImageCLEF2025,
    title = {
        Overview of ImageCLEF 2025: Multimedia Retrieval in Medical, Social
        Media and Content Recommendation Applications},
    author = {
        Ionescu, Bogdan and M\"uller, Henning and Stanciu, Dan-Cristian and
        Andrei, Alexandra-Georgiana and Radzhabov, Ahmedkhan and Prokopchuk,
        Yuri and {\c{S}tefan, Liviu-Daniel} and Constantin, Mihai-Gabriel and
        Dogariu, Mihai and Kovalev, Vassili and Damm, Hendrik and R\"uckert,
        Johannes and Ben Abacha, Asma and Garc\'ia Seco de Herrera, Alba and
        Friedrich, Christoph M. and Bloch, Louise and Br\"ungel, Raphael and
        Idrissi-Yaghir, Ahmad and Sch\"afer, Henning and Schmidt, Cynthia
        Sabrina and Pakull, Tabea M. G. and Bracke, Benjamin and Pelka, Obioma
        and Eryilmaz, Bahadir and Becker, Helmut and Yim, Wen-Wai and Codella,
        Noel and Novoa, Roberto Andres and Malvehy, Josep and Dimitrov, Dimitar
        and Das, Rocktim Jyoti and Xie, Zhuohan and Shan, Hee Ming and Nakov,
        Preslav and Koychev, Ivan and Hicks, Steven A. and Gautam, Sushant and
        Riegler, Michael A. and Thambawita, Vajira and P\r{a}l Halvorsen and
        Fabre, Diandra and Macaire, C\'ecile and Lecouteux, Benjamin and
        Schwab, Didier and Potthast, Martin and Heinrich, Maximilian and
        Kiesel, Johannes and Wolter, Moritz and Stein, Benno
    },
    year = 2025,
    month = {September 9-12},
    booktitle = {Experimental IR Meets Multilinguality, Multimodality, and Interaction},
    publisher = {Springer Lecture Notes in Computer Science LNCS},
    address = {Madrid, Spain},
    series = {
        Proceedings of the 16th International Conference of the CLEF
        Association (CLEF 2025)},
    pages = {}
}

The Organising Committee

Overall coordination

• Bogdan Ionescu <bogdan.ionescu(at)upb.ro>, National University of Science and Technology Politehnica Bucharest, Romania
• Henning Müller <henning.mueller(at)hevs.ch>, University of Applied Sciences Western Switzerland, Sierre, Switzerland
• Cristian Stanciu <stanciu.cristi12(at)gmail.com>, National University of Science and Technology Politehnica Bucharest, Romania

Technical support

• Ivan Eggel <ivan.eggel(at)hevs.ch>, University of Applied Sciences Western Switzerland, Sierre, Switzerland
• Liviu-Daniel Ștefan <liviu_daniel.stefan(at)upb.ro>, National University of Science and Technology Politehnica Bucharest, Romania